What do we mean by partnership ? Creating Take a taxi - PowerPoint PPT Presentation

What do we mean by ‘partnership’ ? Creating 
 Take a taxi Human-Computer Partnerships Driver in control Wendy E. Mackay Inria, Université Paris-Saclay • 21 September 2018 What do we mean by ‘partnership’ ? What do we mean by ‘partnership’ ? Take a taxi Take a taxi Driver in control Driver in control � � Drive a motorcycle Drive a motorcycle User in control User in control � Ride a horse Shared control

How do we interact with computers ? Human-Computer Partnerships Human- Computer as tool Human -in-the-loop Computer Empower users Interaction � Computer as servant 
 Artificial Delegate tasks Intelligence � Computer as medium Mediated Communicate Communication Wait what about this 
 A ‘simple’ human-computer partnership ‘human-in-the-loop’ ? User types – Google suggests – User chooses

Human-Computer Partnerships Human-Computer Partnerships Human -in-the-loop Human-in-the-loop Machine learning perspective: Machine learning perspective: Human helps improve the algorithm Human helps improve the algorithm � Computer -in-the-loop Human-Computer Partnerships Human-in-the-loop Human helps improve the algorithm � Computer -in-the-loop How can we Human-computer interaction perspective: empower people ? Computers empower the human user

We face a major design trade-off Solution: Shift the curve power power How to balance: Simple things power of expression should be simple, complex things 
 simplicity of execution should be possible simplicity simplicity Unified principles of interaction Unified principles of interaction Two complementary perspectives: Two complementary perspectives: System: How to build it ? System: How to build it ? Instrumental Interaction Instrumental Interaction and Substrates and Substrates Human: How to interact with it ? Co-adaptive Systems Human-computer partnerships with Michel Beaudouin-Lafon

Human-computer partnerships What can we learn from physical tools ? People can We can use physical tools as designed… 
 adapt to technology they learn it adapt the technology they appropriate it Discoverability Appropriability Expressivity What can we learn from physical tools ? We can appropriate physical tools But we can also improvise 
 20

We can appropriate physical tools We can appropriate physical tools … why not software ? 21 22 Why can’t we learn to ‘play’ software tools ? 
 without relearning the interface 
 with every software upgrade ? Musical instruments become part of 
 the body

Ways of interacting with computers Human-computer partnerships Human- Computer as tool People can Computer Empower users adapt to technology they learn it Interaction adapt the technology they appropriate it � Computer as servant 
 Artificial � Delegate tasks Intelligence Computers can adapt to people they learn (AI) � Computer as medium adapt people’s behavior they teach Mediated Communicate Communication Human-Computer Partnerships 
 Smartphone interfaces are simple People want: Discoverability Appropriability Expressivity

Smartphone interfaces are simple Discoverability How can I learn 
 How to make them powerful, expressive and simple ? new gestures 
 and commands ? Octopocus Dynamic partnership UIST ‘09 Learn gestures that issue commands Experts just perform the gesture Progressive feedforward 
 Novices pause ... the guide appears What gestures are available ? Progressive feedback 
 What did the system recognize ? Olivier Bau

Octopocus video Appropriability How can I define 
 my own gestures ? Fieldward Fieldward: create personal gesture commands CHI ‘17 Create personal gesture commands Choose easy-to-remember gestures Progressive feedforward reveals whether • command exists • it is recognizable Joe Malloch, Carla Griggio & Joanna McGrenere

Expressive Keyboard 
 CHI‘16 Expressivity Redefining gesture-typing keyboards to support user expression Four ways to type the word “great” all produce the same result How can I 
 express myself ? Jessalyn Alvina, Joe Malloch Expressive Keyboard Expressive Keyboard Machine learning guesses the correct word 
 Gesture variations creates expressive text � Users control 
 text color font style and emojis

Human-Computer Partnerships Expressive typography Interactive paper for musicians Discoverability Appropriability Expressivity Paper Tonnetz SMC‘12 Discoverability Musical relationships create a ‘paper substrate’ Draw music with gestures How can I learn 
 musical relationships ? Jérémie Garcia, Fanis Tsandilas

Paper Tonnetz Appropriability How can I define 
 my own commands ? Knotty Gestures : AVI‘10 Draw a knot to define a command Knots can define Interact now or later 
 mathematical or other relationships Fanis Tsandilas

Knotty Gestures Knotty Gestures Draw a line with a knot Add another knot to define Choose “recording” to define the type of line the start of the recording start rec rec Knotty Gestures Knotty Gestures Add a third knot to define Slide the pen back and forth the end of the recording to play the recording end end start start rec rec

Knotty Gestures Expressivity How can I 
 express myself ? 
 Quid Sit Musicus 
 Paper Composer World Premier 2014 IHM‘14 13th century musical scores Associate gesture characteristics Each note indicates expression with sounds and features � compose & perform Philippe Leroux, Jérémie Garcia Jérémie Garcia


 Quid Sit Musicus? Goal: 
 true human-computer 
 partnerships 
 that empower 
 rather than frustrate 
 (or replace) people