Interaction beyond Computation Michel Beaudouin-Lafon in|situ| - - PowerPoint PPT Presentation

▶
interaction beyond computation
SMART_READER_LITE
LIVE PREVIEW

Interaction beyond Computation Michel Beaudouin-Lafon in|situ| - - PowerPoint PPT Presentation

Feb 07, 2024 332 likes •852 views

Interaction beyond Computation Michel Beaudouin-Lafon in|situ| Universit Paris-Sud & INRIA Stanford University 1 Why this talk? 2 Why this talk? Interaction (not just human-computer interaction) is becoming a key factor of most

slide-1
SLIDE 1

Interaction beyond Computation

Michel Beaudouin-Lafon in|situ| Université Paris-Sud & INRIA Stanford University

1

slide-2
SLIDE 2

Why this talk?

2

slide-3
SLIDE 3

Why this talk?

Interaction (not just human-computer interaction) is becoming a key factor of most computer systems Streaming algorithms, Anytime algorithms, Interactive grids, Cloud computing, Interactive proofs, Service-oriented architectures, etc. Human-computer interaction has stopped “thinking big” Lack of long-term visions Many incremental point designs, Few full-scale explorations, Integrative research

3

slide-4
SLIDE 4

Overview

Interactive Computation Interaction as phenomenon: multi-scale pointing and navigation Interaction as first class object: Instrumental interaction Interaction in the large: the WILD room

4

slide-5
SLIDE 5

Interactive computation

Non-algorithmic computational problem Dynamic interleaving of input & output streams Dependency on the environment Parallel computation of human and computer Non-computability of the environment

5

slide-6
SLIDE 6

Non-algorithmic computational problem

“Models of interaction capture the notion of performing a task or providing a service, rather than algorithmically producing outputs from inputs” Sometimes we rely on the computer being unable to solve the problem:

MillionVis (Fekete, infovis02)

6

slide-7
SLIDE 7

Dynamic interleaving of input and output streams

“Interactions may consist

  • f interleaved inputs and
  • utputs modeled by

dynamic streams; future input values can depend on past output values.” Feedback loops at multiple levels of scale

Octopocus (Bau & Mackay, uist08)

7

slide-8
SLIDE 8

Dynamic interleaving of input and output streams

StreamLiner (Yuan, Tabard & Mackay, kam08)

8

slide-9
SLIDE 9

Dependency on the environment

“In models of interaction, the world or environment of the computation is part of the model and plays an active part in the computation by dynamically supplying the computational system,

  • r agent, with inputs, and

consuming the output values the system produces.”

ABook (Mackay et al., uist01)

9

slide-10
SLIDE 10

Parallel computation of user and computer

“In models of interaction, computation may be concurrent; a computing agent can compute in parallel with its environment and with

  • ther agents.”

Pick and drop (Rekimoto, uist97)

10

slide-11
SLIDE 11

Non-computability

  • f the environment

“The environment cannot be assumed to be static or even effectively computable; for example, it may include humans or other real-world elements.”

11

slide-12
SLIDE 12

Users Computers Artifacts Environment Users Artifacts

What does this tell us?

We need models, frameworks and tools that account for interaction Interaction as phenomenon Interaction as first-class object Integrative research

12

slide-13
SLIDE 13

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law

Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses Wall pointing

MT = k.ID, ID = log2(A/W + 1)

(Fitts, 1954)

13

slide-14
SLIDE 14

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits

Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses Wall pointing

MT = k.ID

(Guiard & Beaudouin-Lafon, hci01)

14

slide-15
SLIDE 15

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits

Effect of view size

Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses Wall pointing

MT = k.ID / V

(Guiard, Beaudouin-Lafon, Bastin, Pasveer & Zhai, avi04)

15

slide-16
SLIDE 16

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits Effect of view size

Orthozoom

Semantic pointing Object pointing Dynaspot Sigma Lenses Wall pointing

(Appert & Fekete, chi06)

16

slide-17
SLIDE 17

Interaction as phenomenon: multi-scale pointing and navigation

17

slide-18
SLIDE 18

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits Effect of view size Orthozoom

Semantic pointing

Object pointing Dynaspot Sigma Lenses Wall pointing

Visual space motor space (Blanch, Guiard & Beaudouin-Lafon, chi04)

18

slide-19
SLIDE 19

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing

Object pointing

Dynaspot Sigma Lenses Wall pointing

(Guiard, Blanch & Beaudouin-Lafon, gi04)

19

slide-20
SLIDE 20

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing

Dynaspot

Sigma Lenses Wall pointing

(Chapuis, Labrune & Pietriga, chi09)

20

slide-21
SLIDE 21

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot

Sigma Lenses

Wall pointing

(Appert & Pietriga, chi08)

21

slide-22
SLIDE 22

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses

Wall pointing

22

slide-23
SLIDE 23

Interaction as phenomenon: multi-scale pointing and navigation

Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses

Wall navigation

(Nancel et al., chi’11)

23

slide-24
SLIDE 24

Multi-scale pointing and navigation: summary

Information transfer with a very tight feedback loop Interaction is directly

  • bservable

Assisted pointing: what information can the system provide to improve pointing performance?

24

slide-25
SLIDE 25

Interaction as first-class object: instrumental interaction

(D’alembert & Diderot, L’Encyclopédie, 1751)

25

slide-26
SLIDE 26

26

slide-27
SLIDE 27

27

slide-28
SLIDE 28

Instrumental interaction

Two levels of interaction: mediation Instrument as extension of one’s body

(Beaudouin-Lafon, chi00)

28

slide-29
SLIDE 29

Instrumental interaction

Covers many interaction styles: Traditional GUI Novel techniques Tangible interaction

29

slide-30
SLIDE 30

Proof-of-concept: CPN2000

Bi-manual interaction, Marking menus, Toolglasses 40,000+ downloads

30

slide-31
SLIDE 31

Instrumental interaction: Design principles

Reification extends the notion of what constitutes an object Polymorphism extends the power of these commands with respect to these objects Reuse provides a wa of capturing and reusing interaction patterns

(Beaudouin-Lafon & Mackay, avi00)

31

slide-32
SLIDE 32

Reification: aligning objects

Align command Align now and forget it Alignment instrument Align, and keep aligned

32

slide-33
SLIDE 33

POLYMORPHISM AND REUSE

Polymorphism Open-Close Cut-Copy-Paste Undo-Redo Color picker Output Reuse Copy-paste Duplicate Input Reuse Redo Macros

33

slide-34
SLIDE 34

Objects Commands Interface Reification Polymorphism Reuse Objects Reuse

Instrumental interaction: Design principles

Reification and polymorphism: more objects but fewer commands Reification facilitates

  • utput reuse

Polymorphism facilitates input reuse

34

slide-35
SLIDE 35

Instrumental interaction: VIGO

Ubiquitous instrumental interaction Separating instruments from objects Instruments that span multiple surfaces

(Klokmose & Beaudouin-Lafon, chi09)

35

slide-36
SLIDE 36

Object Governor Instrument View

Manipulates Observes

Interactions as first class objects: VIGO architecture

Objects = state Governors = rules Instruments = interaction Views = rendering Decoupling and integration

(Klokmose & Beaudouin-Lafon, chi09)

36

slide-37
SLIDE 37

Integrative Research Interaction In-the-large

37

slide-38
SLIDE 38

Integrative Research in HCI

Putting together complete systems Testing validity in the field Working with extreme users Creating software tools

(Xerox Star, 1981)

38

slide-39
SLIDE 39

Integrative research: Metisse

Window system to test windowing techniques with real applications

(Chapuis & Roussel, uist05 + uist06, chi07, chi09)

39

slide-40
SLIDE 40

Metisse: Façades

40

slide-41
SLIDE 41

Integrative research: WILD

41

slide-42
SLIDE 42

Our extreme users: Scientists

42

slide-43
SLIDE 43

43

slide-44
SLIDE 44

Exploring Multisurface Interaction

44

slide-45
SLIDE 45

Interacting with complex data

Navigate Compare Aggregate Communicate

45

slide-46
SLIDE 46

Interacting with complex data

Navigate Compare Aggregate Communicate

46

slide-47
SLIDE 47

Scene graph PaperWall Master facet Wall front-end/VICON controller Papers Multi-touch table Replicated scene graph Table Application facet Collection

  • f papers

Machine 1 Machine 16 Visualiser facet Replicated scene graph PaperWall facet The Wall Instruments Input Devices

Substance : infrastructure for multisurface interaction

Data-oriented model Sharable graph Distributed Instruments Application teleportation (Scotty)

D B B B D D B/D B/D B/D B/D D D D D D B B B B Control Flow Information Flow Behaviour Centric Behavior Oriented D D D D D B B B B B Data Oriented Data Centric

47

slide-48
SLIDE 48

WILD: multisurface interaction

48

slide-49
SLIDE 49

One step beyond: DIGISCOPE

9 interactive rooms interconnected by high-end videoconferencing Applications to scientific discovery, product life-cycle management, decision making, crisis management, training and education

49

slide-50
SLIDE 50

Summary

A paradigm shift: From algorithmic to interactive computation An interaction model: Instrumental interaction - interaction as first-class object Studying interaction at all scales: From low-level phenomena to integrative environments An overarching goal: Generative theories to create the next generation of interactive systems

50

slide-51
SLIDE 51

Questions

51