Explaining smart systems to encourage (or discourage?) user - - PowerPoint PPT Presentation

▶
explaining smart systems to encourage or discourage user
SMART_READER_LITE
LIVE PREVIEW

Explaining smart systems to encourage (or discourage?) user - - PowerPoint PPT Presentation

Sep 27, 2023 355 likes •536 views

Explaining smart systems to encourage (or discourage?) user interaction Dr Simone Stumpf Centre for HCI Design Simone.Stumpf.1@city.ac.uk @DrSimoneStumpf Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf 2 Bio End-user interactions with AI

slide-1
SLIDE 1

Explaining smart systems to encourage (or discourage?) user interaction

Dr Simone Stumpf Centre for HCI Design

Simone.Stumpf.1@city.ac.uk @DrSimoneStumpf

slide-2
SLIDE 2

2 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Bio

§ End-user interactions with AI – explanations, user

experience and trust

§ Projects §Researcher on DARPA-funded ”CALO” project to use machine learning system to track and suggest appropriate task-based resources §Co-I on NSF-funded “End-user debugging of machine-learned programs“ project §PI for the “FREEDOM” project on smart heating system UIs §Co-I on EPSRC ”SCAMPI” project to develop a smart home technology for self-managing quality of life plans for people with dementia and Parkinson’s

slide-3
SLIDE 3

3 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

AI?

slide-4
SLIDE 4

4 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

The old AI is not the new AI

§ “Old” AI worked mainly on rule-based inferences that were

“extracted” by a knowledge engineer e.g. expert systems

§ “New” AI is typically based on machine learning using

complex statistical inferences e.g. SVMs, deep learning

§Usually system learns a function or weights from (large) data sets so it can provide some appropriate output

slide-5
SLIDE 5

5 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

HCI issues for AI

User Experience Intelligibility Controllability

§ “Black boxes” don’t communicate

how they work

§ “Magic” to users so have very poor

mental models of what is going on

§ What happens if AI goes wrong?

slide-6
SLIDE 6

6 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Explanatory debugging [Kulesza et al. 2015]

§ Debugging is trying to

identify and correct mistakes in a system’s program = controllability

§ Explanation-centric

approach to help end users effectively and efficiently personalize machine learning systems = intelligibility

§ Note: explanations can be

words or pictures

Explanation Feedback Improved behaviour Improved mental model

slide-7
SLIDE 7

7 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Principles of explanatory debugging

§ Explainability §Be iterative: small, consumable bites as users interact §Be sound: truthful to how the system works to build trust §Be complete: include as much as possible what it uses §But don’t overwhelm: tradeoff between soundness, completeness and attention § Correctability §Be actionable: users can modify the system based on the explanations §Be reversible: users may make it worse and need a way to back

  • ut

§Always honour feedback: don’t disregard what the user tells the system §Incremental changes matter: need to build up system changes iteratively

slide-8
SLIDE 8

8 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Example system: EluciDebug

slide-9
SLIDE 9

9 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Explanability and correctability

slide-10
SLIDE 10

10 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Evaluation study

§ 77 participants split into two groups: 40 using EluciDebug, 37

using a version without explanations and advanced feedback

§ 20 Newsgroup data set (Hockey and Baseball): initial system

training on 5 messages for each subject, 1850 unlabeled messages to sort

§ 30 minutes to “make the system as accurate as possible” § Measures: accuracy, feedback given, mental model scores,

perceived workload

slide-11
SLIDE 11

11 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Results

§ Better accuracy §85% for EluciDebug versus 77% at end of study § With less feedback §On average, they added 34.5 new features, removed 8.2 features, made 18.3 feature adjustment and interacted with 47 messages (while control users had to label an average of 182 messages) §No difference in workload § With better understanding §15.8 mental model score versus 10.4 §Correlation between mental model score and system accuracy at study end (i.e. the more you understand the better you can make it)

slide-12
SLIDE 12

12 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Not all systems are the same

§ Not all systems should demand high

user interactions

§“Calm technology” [Weiser and Brown 1997] §“Constrained engagement” between the system and user [Yang and Newman 2013] § Not all explanations increase users

correcting the system [Bussone et al. 2015]

§Provide explanations to discourage user interactions?

slide-13
SLIDE 13

13 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Traditional heating systems user controls heat emits heat makes heat

slide-14
SLIDE 14

14 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

smart heating systems emits heat makes heat system controls heat

slide-15
SLIDE 15

15 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

How to discourage user interactions

§ Study on what users want to know

[Skrebe and Stumpf 2017]

§”Unexpected behaviour” e.g. preheating, demand response, overshooting temp, etc §Detailed reasons for decisions and benefits §Notify before action is taken (ideally) for added control § Design of UI §Text first, graph on demand §Explanations fit with Explanatory Debugging (mostly)

Next: IN period starts 18:15

Electricity – heat pump

16.5˚

Indoors Now IN IN

19˚

Preheating to

16.5˚

Now indoors So you'll be comfortable <in the morning>, your home is preheating. It will take <X> hrs <Y> mins to reach <19˚> by <18:45>, based on:
  • The indoor and outdoor temperatures
  • How well your home holds its heat
  • Your Comfort & Savings settings
<Plus, your electricity is currently at a lower rate, so preheating now is better value for you.> Preheating will continue until the start of the next IN period.

OK

Show graph Remove message Edit schedule 1 2 3 4 5 6 Preheating Edit schedule Comfort & Savings
  • Temp. outdoors
  • Temp. indoors
Occupancy period Primary heat source Heat source cost efficiency Preheating Electricity high demand period 17:00 11:00 08:00 05:00 14:00 23:00 05:00 02:00 20:00 System on, warming home Heat pump on low power System off Heat switched to gas boiler Heat switched to heat pump Comfort & Savings settings in effect 6 8 10 12 14 16 18 4 2 20 6 8 10 12 14 16 18 4 2 20 Pre- heating Preheating Pre- heating IN 06:45 – 09:45 19˚ ASLEEP 21:45 – 06:45 12˚ IN 18:15 – 21:45 19˚ ASLEEP 21:45 – 06.45 12˚ Temperature
  • utdoors
Temperature indoors Heat source cost efficiency More expensive Less expensive Electricity – heat pump Gas – boiler OUT 09:45 – 18:15 8˚ £  £  £  6

Phone is forced landscape

7 8 9 10 11 12

Screen 1.2

slide-16
SLIDE 16

16 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Explanation evaluation study

§ 60 participants with simulated home heating scenarios § 4 conditions §Control: no explanation, normal heating UI only 
 §Text-only: written explanation given, no graphical element 
 §Graphical-only: graphical explanation given, no written element 
 §Both: full explanation given, consisting of both graphical and written elements § Explanations increased understanding § Explanations did not increase trust in the system §Without explanations were more concerned about predictability

  • f system actions

§With any of explanations more focused on whether system was doing the right thing

slide-17
SLIDE 17

17 Simone.Stumpf.1@city.ac.uk | @DrSimoneStumpf

Future work

§ Evaluation of smart heating UIs in field trial § Publication on FREEDOM designs and evaluations § Workshop on explainable smart systems (ExSS) at IUI 2018 § Research and dissemination of SCAMPI project results

Big questions: How to design with AI as a new material? How can we craft better explanations for use in different AI systems? What are the impacts of explanations and user interactions with AI?