Situated Interaction (InSitu) Head: Wendy Mackay 27-28 Nov. 2013 - - PowerPoint PPT Presentation

27-28 Nov. 2013 Comité d'évaluation du LRI

• 1
• Head: Wendy Mackay
• Situated Interaction (InSitu)

27-28 Nov. 2013! Comité d'évaluation du LRI ! 2 !

InSitu Group members!

Permanent faculty! !Caroline Appert, CR1 CNRS !Michel Beaudouin-Lafon, PRCE P-11! !Anastasia Bezerianos, MC P-11 !Olivier Chapuis, CR1 CNRS! !Cédric Fleury, MC P-11 !Stéphane Huot, MC P-11! !Wendy Mackay, DR1 INRIA !Theophanis Tsandilas, CR1 INRIA! Ph.D. students! !David Bonnet !Jérémie Garcia! !Ghita Jalal !Alexandre Kouyoumdjian! !Can Liu !Justin Mathew! !Cyprien Pindat !Daniel Strazzula !! !Oleksandr Zinenko! Post-docs ! !Lora Oehlberg !Halla Olafsdottir! Engineers! !Gabriel Tezier !Monireh Sanaei! !Jonathan Thorpe !Amani Kooli!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 3 !

Computers are everywhere!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 4 !

Computer! Capacity!

Computer capacity is increasing…!

Time!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 5 !

Computer! Capacity!

Computer capacity is increasing…!

but human cognitive and sensorimotor skills are not.!

Time! Human!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 6 !

Artificial Intelligence!

2001 A Space Odyssey

Creating Smart Computers

27-28 Nov. 2013 Comité d'évaluation du LRI

• 7
• Human-Computer Interaction

Iron Man

Augmenting Human Capabilities

27-28 Nov. 2013 Comité d'évaluation du LRI

• 8
• Human-

Computer Interaction Artificial Intelligence Social media Multimedia

Three interaction paradigms

Computer as tool

First person interfaces Empower users

• Computer as servant

Second person interfaces Delegate tasks

• Computer as medium

Third person interfaces Communicate

27-28 Nov. 2013! Comité d'évaluation du LRI ! 9 !

Power!

Trade-off!

How can we keep simple things simple …! but enable complex tasks?!

Simplicity!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 10 !

Power!

Research Challenge!

How can we move the curve?!

Simplicity!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 11 !

Scientific Themes & Objectives!

InSitu emphasizes situated interaction ! ! We develop theory, methods and technology that! !• generate new forms of interaction, ! !• significantly empower users, and! !• inspire the next generation of interactive systems.!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 12 !

Positioning!

France!

!Closest groups: Grenoble, Toulouse, Lille!

Europe!

!UK Labs (Lancaster, RCA, Glasgow)! !Scandinavian labs (KTH, Aarhus)! !German labs (Aachen, Hasso Platner)!

North American Universities!

!Stanford, MIT, Carnegie Mellon,! !U. Toronto, UCSD, Georgia Tech! !

Corporate labs!

!Microsoft (Redmond & Cambridge), Google!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 13 !

InSitu Research Themes!

Interaction and visualization paradigms! !Manage the trade-off between power and simplicity! Mediated communication! !Establish peripheral and focused awareness at a distance! Research methods! !Engage users throughout the design process! Engineering of interactive systems!

!Create tools and platforms for advancing the state of the art!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 14 !

design experiment! build system! develop working ! prototypes! no standard benchmarks!

• perationalize!

run experiment! extract key features! controlled lab studies,! longitudinal field studies! develop theory! analyze results! Fitts’ law, human ! perception! quantitative and qualitative! get an idea! frame paper! participatory design to! avoid toy problems! key insights to generalize!

HCI Research Process!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 15 !

Research Process!

evaluation!

• bservation!

new! evaluation! model! new ! model! new ! model ! Real world! Theory!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 16 !

Research Process!

evaluation!

• bservation!

new! evaluation! model! new ! model! new ! model ! Artifact! prototype! prototype! system! Real world! Theory!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 17 !

Research Contributions!

Fundamental research! !investigate details of human behavior! Invention! !design & benchmark novel interaction techniques! Real-world exploration! !participatory design with users in real-world settings! Innovation! !transfer our technology! !

27-28 Nov. 2013! Comité d'évaluation du LRI ! 18 !

Interaction and visualization paradigms!

Interaction on the desktop:
 Manage the trade-off between power and simplicity!

DynaSpot! JellyLens! Octopocus! Dwell-and-Spring! Combine area and point cursor to improve performance! Add undo to! direct manipulation,! e.g., drag ‘n drop! ! Learn complex gestures with a dynamic guide ! Adapt dynamically 
 to object’s shape for! better focus+context!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 19 !

Example: JellyLenses!

Contribution: Invention!

Take advantage of an underlying representation! to improve the trade-off between focus and context ! ! JellyLenses !

• adapt dynamically!

to the object’s shape!

• optimize placement of !

focus, context & transition ! regions!

• consistently outperform !

fisheye lenses! ! ! !

27-28 Nov. 2013! Comité d'évaluation du LRI ! 20 !

Interaction and visualization paradigms!

Interaction off the desktop
 Manage the trade-off between power and simplicity!

Musink*! WILD pan & zoom! BiPad! EWE! Augment paper scores with user- defined annotations! Eyes-free entry of commands with rhythmic input ! Bimanual tablet interaction: taps, chords & gestures! Compare 12 wall-size interaction techniques across 3 dimensions!

*Zoom on interactive paper!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 21 !

WILD Pan & Zoom!

Contribution: Invention!

Create and evaluate 12 multi-scale navigation techniques! 3 Factors: uni/bimanual, guidance & movement type! Identify individual and interaction effects!

Nancel, Wagner, Pietriga, Chapuis & Mackay (2011) CHI ’11 (Best Paper)!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 22 !

Mediated Communication!

Establish peripheral and focused awareness ! !at a distance!

WeMe! Marker Clock! CHI’13 Scheduler! Multi-scale mediated communication ! Collaborative constraint solving with crowd-sourced data, 1000+ events! Elderly peer-care via peripheral awareness ! Ambient & intentional sharing! Pêle-Mêle!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 23 !

Example: ICI-TV!

Contribution: Innovation!

Research from the EU InterLiving project on ! !mediated communication led to the ICI-TV OMTE! !project with Presto, a CEA startup! ! Praesto provides services! to the elderly via a ! television set-top box ! Emphasizes ! bilateral communication! to help the elderly! age in place! ! ! !

27-28 Nov. 2013! Comité d'évaluation du LRI ! 24 !

Research Methods!

Engage users throughout the design process! !with participatory design!

Generative Walks! Touchstone! Inksplorer! Prism! Structured walk-
 throughs of socio- technical principles ! 7-month field study!

• f a hybrid paper-

electronic notebook! Support transition between paper and electronic exploration! Evaluate trade-offs and run controlled! experiments!

27-28 Nov. 2013 Comité d'évaluation du LRI

• 25
• Example: Prism

Contribution: Real-world exploration

Participatory design with biologists (Institut Pasteur, INSERM, INRA)

• Design of a hybrid interactive paper/electronic laboratory notebook

Field study with INRA biologists (7 months)

• Tabard, Mackay & Eastmond (2008) ACM CSCW'08

27-28 Nov. 2013! Comité d'évaluation du LRI ! 26 !

Engineering of Interactive Systems!

Create tools and platforms that support our research ! !and advance the state of the art! !

FlowStates! JBricks! Substance! Toolkit that combines data flows & state machines! Middleware for multi- surface applications with diverse input! Platform for zooming through massive datasets

• n a wall display!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 27 !

FlowStates!

Contribution: Fundamental research!

!Combine two computational models
 to increase power of expression! ! !Data Flow: ! ! ! !State Machines:! !Low-level event streams ! !Higher-level interactions! ! !! ! !

! !

Appert, Huot, Dragicevic & Beaudouin-Lafon, IHM’09 (Best paper)!

Tracking! Menu

27-28 Nov. 2013! Comité d'évaluation du LRI ! 28 !

Demos

Gimlense: a multi-view, detail-in-context visualization technique that lets users navigate complex 3D models by interactively drilling holes into their outer layers to reveal objects that are buried, possibly deep, into the scene.

Gliimpse: a quick preview technique that smoothly transitions

between document markup code (HTML, wiki markup or LATEX) and its visual rendering. It supports rapid overviews of code-to- document mappings, saving screen real-estate.

WILD interactive visualization of CHI 2013 schedule:

(500 presentations in 200 sessions, with 13 parallel tracks over four days). Abstracted into a simpler classification task for an experiment to compare desktop and wall-sized interaction.

27-28 Nov. 2013! Comité d'évaluation du LRI ! 29 !

Scientific Production

Publications! !Journals !22 major + 3 minor ! !Conferences !77 !(41 major** + 36 mid-level) !! !Book Chapters !7! !Edited Proceedings !5 !

!

Contracts !6,343 M€! !! Keynotes !5! Invited Talks !81! Software & Toolkits !13!

! ** !A+ ACM/SIGCHI !23 ACM/CHI, 7 ACM/UIST, 1 ACM/CSCW!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 30 !

Teaching!

Theses !11!defended! Habilitations ! 2 defended! ! Created 2 International HCI Masters programs Master Research HCID Professional (ICT Labs) Teaching in 16 Master-level courses Master’s Research seminar (CS department) Ph.D. Seminar Post-thesis research careers MIT, Disney/CMU, Microsoft Research, U. Munich,

• U. Copenhagen, Alcatel Lucent, Samsung research

Telecom Paris-Tech, U. Lyon, U. Grenoble !!

27-28 Nov. 2013! Comité d'évaluation du LRI ! 31 !

Honors and Visibility

Honors CHI academy Gilles Kahn thesis prize 9 Best Paper or Honorable Mention Awards ERC advanced grant IUF Senior Marie Curie grant (with MIT) Academic exchanges Muse: Equipe Associée with U. Toronto SIRIUS: Equipe Associée with Stanford, UC San Diego, Berkeley Chile ALMA radio-telescope

27-28 Nov. 2013! Comité d'évaluation du LRI ! 32 !

Diffusion & Professional Service

Conference organization !ACM/CHI’13 in Paris! ! !6 days, 1000+ events, 3500 attendees! Editorial Boards !7 (all international)! Program Chairs !12 (10 international)! Program Committees !52 (46 international)! ! Interviews !France Culture, France Inter, France Bleu, 


!La Recherche, Science & Vie,! ! !Forum Le Monde La Recherche!

! Fêtes de la Science !3 organizers! Demos !uncountable…! ! Transfer to startups !2 (ICI-TV, CEA | Buena Vista, SDS)!

27-28 Nov. 2013 Comité d'évaluation du LRI

• 33
• Head: Wendy Mackay

Situated Interaction (InSitu)

27-28 Nov. 2013! Comité d'évaluation du LRI ! 34 !

Highlights

Prizes and Honors 9 paper awards, 1 thesis prize 2 ACM/CHI academy members, I IUF member Major Grants ERC Advanced, Digiscope EquipEx, Microsoft-Inria Marie Curie, ANR Jeunes Chercheurs, Digiteo Organization of CHI 2013 in Paris Teaching 2 International HCI Masters Master’s Research seminar

Interactive paper for music composition

Jérémie Garcia Theophanis Tsandilas Wendy Mackay

InSitu

Motivation

27-28 Nov. 2013 Comité d'évaluation du LRI 2

Musical score by K. Penderecki OpenMusic, |RCAM

Composition process

27-28 Nov. 2013 Comité d'évaluation du LRI 3

Paper Software Expression Exploration Execution Time

Finale OpenMusic Max/MSP final scores early sketches

Garcia J., Tsandilas T., Agon C., and Mackay W. Inksplorer: Exploring Musical Ideas on Paper and Computer, NIME 2011

Goal

Design interactive tools that support this process from the early most creative phase

27-28 Nov. 2013 Comité d'évaluation du LRI 4

Challenge

Each composer has a very personal composition style How to address composers’ diverse strategies?

27-28 Nov. 2013 Comité d'évaluation du LRI 5

Research methodology

27-28 Nov. 2013 Comité d'évaluation du LRI 6

experiments quasi-experiments public demonstrations field studies

Design Understand users Evaluate

participatory design prototyping toolkits interviews technology probes exploratory studies

Musink [CHI09, Best Paper Award]

27-28 Nov. 2013 Comité d'évaluation du LRI 7

Writing over a paper score with a personal music vocabulary Interactive tool for defining the syntax and recognition of new gesture vocabularies

Paper Substrates [CHI12]

27-28 Nov. 2013 Comité d'évaluation du LRI 8

Paper-based modules representing composition rules and functions Layered paper components that define rhythms, pitch and spatialization

Participatory design with composers

Paper Substrates [CHI12]

27-28 Nov. 2013 Comité d'évaluation du LRI 9

Substrates Toolkit

Interactive, modular paper components specialized to specific types of musical data

Interactive prototypes

Applications

27-28 Nov. 2013 Comité d'évaluation du LRI 10

Substrates for specialized notation PaperTonnetz [Demonstrated at the Palais de la Découverte and CHI13] Fête de la Science, 2011

Evaluation - Polyphony

27-28 Nov. 2013 Comité d'évaluation du LRI 11

Controlled study of composition with 12 composers

Condition A: Graphics Tablet Condition B: Interactive Paper

Research & creation project

With composer Philippe Leroux to compose a commissioned piece concert scheduled for June 27, 2014 at IRCAM

27-28 Nov. 2013 Comité d'évaluation du LRI 12

Research & creation project

27-28 Nov. 2013 Comité d'évaluation du LRI 13

Research & creation project

27-28 Nov. 2013 Comité d'évaluation du LRI 14

Original score

Research & creation project

27-28 Nov. 2013 Comité d'évaluation du LRI 15

Redrawing shapes over the original score

Main publications

27-28 Nov. 2013 Comité d'évaluation du LRI 16

Musink [Tsandilas et al., CHI09] InkSplorer [Garcia et al., NIME11] Paper Substrates [Garcia et al., CHI12] PaperTonnetz [Garcia et al., SMC12, EA CHI13]

27-28 Nov. 2013 Comité d'évaluation du LRI 1

From in|situ| to HCC

Michel Beaudouin-Lafon

27-28 Nov. 2013 Comité d'évaluation du LRI 2

In|situ| assessment

Created in 2002, rapid growth International group, outstanding visibility Spun off three groups Achieved or surpassed most goals Beyond the scientific results, contributed to the visibility of HCI in France

27-28 Nov. 2013 Comité d'évaluation du LRI 3

What’s next?

InSitu will reach the 12-year limit for Inria groups in 2014 Opportunities: Evolution of the field of HCI Paris-Saclay Campus Strategy: more focused research within a strong ecosystem

27-28 Nov. 2013 Comité d'évaluation du LRI 4

Evolution of HCI

Towards physical computing Digital fabrication, Mixed reality, Internet of Things, Towards social computing Social networks, Crowdsourcing, Collective intelligence, More than ever, understand how people adopt, adapt and adapt to technology

27-28 Nov. 2013 Comité d'évaluation du LRI 5

Paris-Saclay

Unique opportunity to be the epicenter of HCI research in Paris-Saclay and in Île-de-France Digiscope / Digipods projects Interaction specialty in the Paris-Saclay Master in Computer Science European HCID Masters (EIT ICT Labs)

27-28 Nov. 2013 Comité d'évaluation du LRI 6

Human-Centered Computing

The HCC group will be composed

• f two more focused research groups

(and maybe more in the future) One around the CREATIV ERC Grant Towards Human-Computer Partnerships One tentatively called ILDA Interacting with Large DatA

27-28 Nov. 2013 Comité d'évaluation du LRI 7

CREATIV – Wendy Mackay

Creating Human-Computer Partnerships Faculty:

Michel Beaudouin-Lafon, Cédric Fleury, Stéphane Huot, Fanis Tsandilas

Co-adaptive instruments

Generative design of co-adaptive systems Participatory design w/ creative professionals Real-time interaction and interaction streams Personal language creation Reusable activity patterns

27-28 Nov. 2013 Comité d'évaluation du LRI 8

ILDA – Emmanuel Pietriga

Interaction with Large DatA Faculty: Caroline Appert,

Anastasia Bezerianos, Olivier Chapuis

Data-centric interaction

Interaction with structured data Multiscale navigation Collaborative interaction Visual perception Application domains: Critical systems & Scientific data

27-28 Nov. 2013 Comité d'évaluation du LRI 9

Digiscope

Equipex – 22M total, 6.7M from ANR Interaction “in-the-large”: 9 interconnected interactive rooms for collaborative visualization of complex data

27-28 Nov. 2013 Comité d'évaluation du LRI 10

Digipods

Funded by Région Île-de-France Add interactive capabilities to Digiscope Digicarts

“Interaction cloud”

Interactive paper & pen 3D motion capture Webcam Haptic input Dock for laptop Cart on wheels Loudspeakers High-resolution multitouch screen Keyboard and mouse Documents Multitouch

• verlay screen

27-28 Nov. 2013 Comité d'évaluation du LRI 11

HCC over-arching goals

Understanding fundamental aspects of human behavior in interactive situations Designing novel interaction and visualization methods Developing design methods that harness user innovation Creating advanced tools for creating interactive systems

27-28 Nov. 2013 Comité d'évaluation du LRI 12

Other goals for HCC

Develop HCI in teaching programs Develop partnerships with design schools and design research groups Industry transfer Paris-Saclay HCI Institute?