Mobile Technologies context and task theory interaction techniques - - PowerPoint PPT Presentation

▶
mobile technologies
SMART_READER_LITE
LIVE PREVIEW

Mobile Technologies context and task theory interaction techniques - - PowerPoint PPT Presentation

Apr 19, 2023 97 likes •472 views

Mobile Technologies context and task theory interaction techniques in/output technologies 1 LMU Mnchen Medieninformatik Andreas Butz, Julie Wagner Mensch-Maschine-Interaktion II WS2014/15

slide-1
SLIDE 1

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Mobile Technologies

1

context and task

theory interaction techniques in/output technologies

Thursday 27 November 14

slide-2
SLIDE 2

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Sensors

  • gyroscope
  • accelerometer
  • magnetometer
  • always available mobile input

– SixthSense project – EMG (Skinput)

2

Thursday 27 November 14

slide-3
SLIDE 3

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

SixthSense Project

  • vision-based approach for always available

input

3

https://www.youtube.com/watch?v=JnxGUPzP_U0

Literature: Mistry, P . : Wear Ur World - A Wearable Gestural Interface, CHI 2009

Thursday 27 November 14

slide-4
SLIDE 4

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Biosensors

  • signals traditionally used for diagnostic

medicine

– emotional state: heart rate, skin resistance – brain sensing technology cognitive and emotional state (BCI - brain computer interaction):

  • electrocephalography (EEG)
  • functional near-infrared spectroscopy (fNIR)

– signals generated by muscle activation

  • electromyography (EMG)
  • bone conduction

– sound frequencies propagate well through bone

4

Thursday 27 November 14

slide-5
SLIDE 5

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Brain-Computer Interfaces

5

contact Mariam.Hassib@ifi.lmu.de

Thursday 27 November 14

slide-6
SLIDE 6

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

The *annoyingly* talking window

6

https://www.youtube.com/watch?v=azwL5eoE5aI

Thursday 27 November 14

slide-7
SLIDE 7

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Skinput

  • Bio-sensing approach

– leverages natural acoustic conduction properties

  • f the human body

7

https://www.youtube.com/watch?v=g3XPUdW9Ryg

Thursday 27 November 14

slide-8
SLIDE 8

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Head-Mounted Display

  • Principles

– closed HMD – VR with HMD – AR with see-through

  • video
  • optical

– google glasses

  • Head-up Displays with head tracking
  • oculus rift

8

Thursday 27 November 14

slide-9
SLIDE 9

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Google Glasses

  • c

Ivan Sutherland, 1968

http://louislau91.files.wordpress.com/2013/06/sword_of_damocles.jpeg

Google glasses and augmented reality

http://www.tomsguide.com/us/ google-glass,news-17711.html

9

Thursday 27 November 14

slide-10
SLIDE 10

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Principle: closed (video only) HMD

  • Monitor is mounted very close to the eye
  • Additional lens makes it appear distant
  • è all images appear at the same distance

– Usually at infinity or slightly less

10

Thursday 27 November 14

slide-11
SLIDE 11

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Creating VR with a HMD

Rendering Head tracker 3D scene

11

Thursday 27 November 14

slide-12
SLIDE 12

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Challenges with HMDs in VR

  • Lag and jitter between head motion and

motion of the 3D scene

– Due to tracking à predictive tracking – Due to rendering à nowadays mostly irrelevant

  • Leads to different motion cues from

– Eye (delayed) and – Vestibular system (not delayed)

  • Result: cyber sickness

12

Thursday 27 November 14

slide-13
SLIDE 13

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Oculus Rift

  • costom-built motion and orientation sensor

unit

– gyroscope – accelerometer – magnetometer

  • head tracking with 3 degrees of freedom

13

http://static3.businessinsider.com/image/ 51d1949aecad04ad3e000000/heres-what-happened-when-we- strapped-a-bunch-of-people-into-the-oculus-rift-virtual-reality- headset.jpg

Thursday 27 November 14

slide-14
SLIDE 14

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Creating AR with video see-through HMDs

Rendering Head tracker 3D scene Video Mixing Parallax error

14

Thursday 27 November 14

slide-15
SLIDE 15

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Advantages of video-based see- through

  • Lag between physical and virtual image

can be compensated

  • Camera can be used for tracking as well

–Physical image = raw tracking data –Perfect registration possible

  • Video mixer can add or subtract light

–Virtual objects can be drawn in black –Physical objects can be substituted –Virtual objects can be behind physical

  • bjects
  • Just one image with a given focus

distance

15

Thursday 27 November 14

slide-16
SLIDE 16

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Challenges of video-based see- through

  • Lag between physical and virtual image

can be compensated

– …by delaying the physical image – Leads back to the cyber sickness problem

  • Parallax error can not be corrected

electronically

– Wrong stereo cues when used for stereo

  • Richness of the world is lost

– Video image just 0.5 megapixels – Resolution of human vision is much higher (>10x)

16

Thursday 27 November 14

slide-17
SLIDE 17

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Creating AR with optical see-through HMDs

Rendering Head tracker 3D scene

17

Thursday 27 November 14

slide-18
SLIDE 18

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Advantages of optical see-through HMDs

  • Preserve the richness of the world

– Very high resolution of physical image – No lag between motion and phys. image – Physical objects can be focused at their correct distance

  • Limitations:

– can only add light, i.e. not cover things – lag of digital image very noticeable

18

Thursday 27 November 14

slide-19
SLIDE 19

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Example: Google glass

19

http://www.youtube.com/watch?v=Ee5JzKbOAaw

Thursday 27 November 14

slide-20
SLIDE 20

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Creating AR with Head-up Displays (HUDs)

Rendering Head tracker 3D scene

20

Thursday 27 November 14

slide-21
SLIDE 21

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Head-Up Display with 3D registration

  • Currently mostly military use
  • Fixed Display
  • Very exact head or eye tracking needed

– Easy for jet pilots

  • High brightness and dynamics needed

21

Thursday 27 November 14

slide-22
SLIDE 22

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

HUD without 3D registration

  • Optional Equipment in premium cars

(image source: www.bmw.ch)

  • easy: no tracking needed! -> not AR!

22

Thursday 27 November 14

slide-23
SLIDE 23

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

HUD app for iPhone

  • can be bought from app

store

  • put iPhone under wind

shield

  • uses GPS and accell

sensors to sense car motion

  • can display speed,

heading, ...

23

Thursday 27 November 14

slide-24
SLIDE 24

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Fabrication, know your tools...

  • know your tools...
  • further development of

– Laser-cutting – 3D printing

24

Thursday 27 November 14

slide-25
SLIDE 25

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Laser Origami

  • laser cut objects without assembling...

https://www.youtube.com/watch?v=arjRtCjI9AQ

Literature: Müller, S. : Laserorigami: Laser-cutting 3D Objects, CHI 2013

25

Thursday 27 November 14

slide-26
SLIDE 26

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

WirePrint

https://www.youtube.com/watch?v=Ea4V7kb2VsY

Literature: Müller, S. : WirePrint:3D Printed Previews for Fast Prototyping, UIST 2014

26

Thursday 27 November 14

slide-27
SLIDE 27

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

3Doodler

  • next step to 3D printing... 3D sketching

https://www.youtube.com/watch?v=DQWyhezIze4

27

Thursday 27 November 14

slide-28
SLIDE 28

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Teach and Learn...

  • Goal: support creative activities of children

using educational toys.

– tangibles – programming by demonstration concept

28

Thursday 27 November 14

slide-29
SLIDE 29

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Sifteo Cubes

https://www.youtube.com/watch?v=fEqq8JykQoQ

Literature: Merrill, D. : Sifteo Cubes, CHI 2012

29

Thursday 27 November 14

slide-30
SLIDE 30

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Programming by

Topobo System

https://www.youtube.com/ watch?v=50JdK_K2NWk

Literatur: Raffle, H. : Topobo: a constructive assembly system with kinetic memory, CHI 2004

30

Thursday 27 November 14

slide-31
SLIDE 31

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Squishy Circuits

Literatur: Johnson, S. : Squishy circuits: A tangible medium for electronics education, CHI 2010

31

Thursday 27 November 14

slide-32
SLIDE 32

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Shape changing displays

  • MorePhone: flexible displays

– using shape deformation as its primary means of both haptic and visual notifications – use OLED and shape changing alloy wires

32

Thursday 27 November 14

slide-33
SLIDE 33

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

MorePhone

33

Literatur: Gomes, A. : MorePhone: An Actuated Shape Changing Flexible Smartphone, CHI 2013

Thursday 27 November 14

slide-34
SLIDE 34

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide 34

OLED - organic light-emitting diode

  • applied: PDAs, photo-camera, phones
  • elements: two electrodes (one of them

transparent), layer of OLED-material

  • idea:
  • ‘+’: thin construction allows fabrication of

flexible displays on e.g. plastic foil, no backlight, higher contrast ratio

  • ‘-’: not all colors shine with same efficiency,
  • n-going research on optimum OLED-

materials

http://www.igm.uni-stuttgart.de/forschung/arbeitsgebiete/oled/index.en.html http://www.blogcdn.com/www.engadget.com/media/2009/01/sony-oled-top002.jpg

Thursday 27 November 14

slide-35
SLIDE 35

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Shape Memory alloy wires

  • shape memory effect

– jumps back into ‘programmed shape’ when heat energy is applied

  • two-way shape memory effect

– springs into other shape when cooled down.

35

https://www.youtube.com/watch?v=JKBM9my5eOA

Thursday 27 November 14

slide-36
SLIDE 36

Mobile context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile projection

LMU München — Medieninformatik — Andreas Butz, Julie Wagner — Mensch-Maschine-Interaktion II — WS2014/15 Slide

Shape Memory alloy wires

  • Flexinol muscle wire

– simple to hook up to a circuit – get inspired by the field of robotics (e.g. artificial muscles)

36

https://www.youtube.com/watch?v=VfOsPAkvOd0

Thursday 27 November 14