Instrumented Environments Andreas Butz, butz@ifi.lmu.de, - - PowerPoint PPT Presentation

▶
instrumented environments
SMART_READER_LITE
LIVE PREVIEW

Instrumented Environments Andreas Butz, butz@ifi.lmu.de, - - PowerPoint PPT Presentation

Nov 22, 2023 222 likes •764 views

Instrumented Environments Andreas Butz, butz@ifi.lmu.de, www.mimuc.de Fri, 12:15-13:45, Theresienstr. 39, Room E 045 15. Mai 06 LMU Mnchen, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 1 Topics today Displays

slide-1
SLIDE 1
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 1

Instrumented Environments

Andreas Butz, butz@ifi.lmu.de, www.mimuc.de Fri, 12:15-13:45, Theresienstr. 39, Room E 045

slide-2
SLIDE 2
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 2

Topics today

Displays

  • small, med, large
  • digital ink, e-paper
  • projection, multiple & steerable
  • Fog screen
  • Spatial audio

Sensing

  • touch screens/input
  • Cameras, microphones
  • Force, accelleration
slide-3
SLIDE 3
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 3

Displays

slide-4
SLIDE 4
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 4

Tabs, pads and boards

(the Xerox ParcTab project)

Tabs Pads Boards

slide-5
SLIDE 5
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 5

Tabs, pads...

Tabs, inch-sized (1 Inch = 2.54 cm)

  • small handheld networked devices

See also Active badges

  • specialized tabs, enable localization

Pads, foot-sized (1 Foot = 30.47 cm)

  • mixture of laptop, palmtop, sheet of paper

Introduced the concept of a disposable computer, no identity, impersonal Provide a solution to the lack of space on windows based systems

slide-6
SLIDE 6
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 6

...and boards

Boards, yard-sized (1 Yard = 0.914 m)

  • used as chalk boards, TVs, display boards

Power of Ubicomp stems from the interaction of all devices. Ubicomp can „awake“ lifeless things (books,

  • verhead slides, etc.)

Problem: today it‘s easier to read a book than to sit down at a complicated Personal Computer Transition will happen in small steps

slide-7
SLIDE 7
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 7

More ubicomp displays

retinal displays clip-on

slide-8
SLIDE 8
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 8

Retinal displays

slide-9
SLIDE 9
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 9

Electronic-ink

slide-10
SLIDE 10
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 10

Projectors

Key Criteria

  • Resolution
  • Brightness
  • Weight
  • Noise
  • Lens
  • Image correction
  • Projection distance
  • Connections
  • Lamp life time
slide-11
SLIDE 11
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 11

CRT projector

Use R,G+B CRTs as light sources Good black areas Low brightness Fast Need to calibrate convergence!

www.projektoren-datenbank.com/rohre.htm

slide-12
SLIDE 12
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 12

LCD projector

www.projectorpoint.co.uk/projectorLCDvsDLP.htm www.projektoren-datenbank.com/lcd.htm

slide-13
SLIDE 13
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 13

DLP projector

slide-14
SLIDE 14
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 14

DLD projector (movie)

http://www.dlp.com/

slide-15
SLIDE 15
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 15

Technological side effects

Screen door effect

  • Caused by LCDs
  • Less prominent in DLP

If a DLP projector is moved, color seams appear DLP principle can be abused for creating imperceptible structured light

A A A A

slide-16
SLIDE 16
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 16

Lens shift

Optical construction No loss of resolution

slide-17
SLIDE 17
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 17

Keystone correction

Computed correction Loss of resolution!

slide-18
SLIDE 18
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 18

Smart Projectors

[Oliver Bimber et al., IEEE Computer, January 2005]

  • Projection onto curved surfaces can be solved by 3D rectification, …but:
  • What if the projection surface is not uniformly colored?
  • See Video (scientific) or Video (TV)
slide-19
SLIDE 19
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 19

Multiple Projector setups

Several problems:

  • Overlapping projection areas
  • No color equivalence between two projectors

(manufacturing & temperature color drift)

  • Minimize sensitivity to small errors in

calibration parameters or mechanical variations

Relatively good solution: Feathering

slide-20
SLIDE 20
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 20

Feathering

Normally the overlap region is a well- defined contiguous region Intensity of every pixel weighted proportional to Euclidian distance to nearest boundary pixel of image Weights in range [0,1] multiplied with intensities in the final image

slide-21
SLIDE 21
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 21

Feathering

If both projectors produce same color, A+B at maximum and constant over surface If not A+B´ produces smooth transition

slide-22
SLIDE 22
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 22

Luminance Attenuation Map

[Majumder & Stevens, VRST 2002] Large display wall with 5x3 projectors Linear ramps (feathering) don‘t work perfectly Goal: get rid of the remaining unevenness Strategy: don‘t assume, but measure!

slide-23
SLIDE 23
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 23

Calibration step

  • Measuring the Luminance Response: The luminance response of

any pixel is defined as the variation of luminance with input at that

  • pixel. We measure the luminance response of every pixel of the

display with a camera.

  • Finding the Common Achievable Response: We find the

common response that every pixel of the display is capable to

  • achieving. The goal is to achieve this common achievable response

at every pixel.

  • Generating the Luminance Attenuation Map: We find a

luminance attenuation function that transforms the measured luminance response at every pixel to the common achievable response.

slide-24
SLIDE 24
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 24

Measured luminance response

Gives a factor for multiplication of the final images (just as in feathering) Can be done in graphics hardware via alpha channels

slide-25
SLIDE 25
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 25

LAM: results

slide-26
SLIDE 26
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 26

PixelFlex2

[Raij, Gill, Majumder,Towles, Fuchs, ProCams 2003]

Uneven brightness and arbitrary geometry:

  • Rectify each projector by calibrating 4 points
  • Used LAMs for brightness
slide-27
SLIDE 27
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 27

PixelFlex2

slide-28
SLIDE 28
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 28

Barco PowerWall

(as seen at the Univ. of Konstanz)

slide-29
SLIDE 29
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 29

Barco PowerWall (back side)

slide-30
SLIDE 30
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 30

PowerWall Specs

Size: 5,20m x 2,15m 4640 x 1920 Pixels 8 computers 8 synchronized projectors „Soft-Edge-Blending“ in the projectors Stereo display with shutter glasses Must not be touched 8( Price: ?00.000,00

slide-31
SLIDE 31
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 31

Everywhere Display Projector (IBM)

http://www.research.ibm.com/ed/ Claudio Pinhanez

slide-32
SLIDE 32
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 32

Everywhere display (cont.)

Output: a projector and a rotating mirror Input: a camera for interaction, NOT for image rectification!

slide-33
SLIDE 33
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 33

Undistorting the projected image

Place original image in the 3D model Virtual camera image shows it distorted Project the distorterd image from 3D model with the Real projector into the real world

– Distortions cancel each other out IF virtual camera and real projector are in the same location

slide-34
SLIDE 34
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 34

Everywhere display (cont.)

Correct distortions

  • Use the fact that camera

and projectors are geometrically the same (optically inverse)

Use standard HW components

  • 3D-Graphics board and

VRML-world

slide-35
SLIDE 35
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 35

Everywhere display (cont.)

BLUESPACE office scenario

slide-36
SLIDE 36
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 36

Everywhere display (cont.)

Collaborative experience at SIGGRAPH 2001

1 5 4 3 2

Video

slide-37
SLIDE 37
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 37

Everywhere display (cont.)

Other Applications

http://www.research.ibm.com/ed/

slide-38
SLIDE 38
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 38

Steerable projector/camera unit

(as used in the FLUIDUM instrumented room)

Projector (3.000 ANSI lumen, XGA resolution) Digital camera (4 megapixels)

  • Also yields live video stream

Moving yoke

  • Used for spotlights on stages
  • Steerable via DMX bus
  • Pan 340° / tilt 270°
  • Ca. 1 rotation in 1s
  • Mechanical precision:

+/- 1cm on the wall

slide-39
SLIDE 39
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 39

Portable Display Screen

[Borokovski, Crowley, INRIA]

slide-40
SLIDE 40
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 40

Roomware

Streitz et al., FhG

slide-41
SLIDE 41
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 41

Connectable Displays

Single usage Connected usage Streitz et al., FhG

slide-42
SLIDE 42
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 42

FogScreen

  • A wall sized,

immaterial projection display

  • Image projected on

dry, non-hazardous fog (pure water)

  • Inventor: Ismo

Rakkolainen

slide-43
SLIDE 43
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 43

FogScreen (2)

Fog sandwiched between airstreams Immaterial user can reach

  • r walk through

unhindered

slide-44
SLIDE 44
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 44

FogScreen (3)

Technical Details

  • First introduced at UIST

2004

  • So far only 9

FogScreens in the world

  • Price: 100k $
  • Weight: around 100 kg
  • Needs: 10L/h, 300W
slide-45
SLIDE 45
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 45

FogScreen (4)

Pseudo 3D

  • Double sided projection possible
  • E.g. opposite views of the same scene
slide-46
SLIDE 46
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 46

FogScreen(4)

Pseudo 3D – HeadTracking (S. DiVerdi)

  • Tracking the users Head
  • Input used for adapting the frustum for

accurate perspective rendering

slide-47
SLIDE 47
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 47

FogScreen (5)

Interaction

  • Laser Range Scanner
  • UltraSound
  • Infrared LED Tracking
slide-48
SLIDE 48
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 48

FogScreen(6)

Interaction – Setup: either

  • LRF attached to the FogScreen corner OR
  • UltraSonic Device attached to FogScreen corner OR
  • Infrared Cameras in the corners of the room
slide-49
SLIDE 49
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 49

Resources

  • http://www.fogscreen.com/
  • http://ilab.cs.ucsb.edu/projects/ismo/fogs

creen.html

  • http://www.worldviz.com/ppt/index.html
  • http://www.robosoft.fr/SHEET/02Local/10

04SickLMS200/SickLMS200.html

  • http://www.e-beam.com/
slide-50
SLIDE 50
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 50

Spatial Audio Cues

IATD

IATD: Interaural time difference: sound arrives at the ears at different times IAID: Interaural intensity difference: sound arrives with different volume Spectral difference; the shape

  • f the head and ear cause a

different spectral transmission to both ears. This can be expüressed in Head-Related Transfer Functions (HRTF)

slide-51
SLIDE 51
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 51

Tracked head phones

[e.g., http://listen.imk.fraunhofer.de/]

slide-52
SLIDE 52
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 52

Vector Based Amplitude Panning

1 2 2 1 1

= = + =

L p g g L l g l g p

T

T

p L g g L l g l g l g p

1 3 3 2 2 1 1

= = + + =

slide-53
SLIDE 53
  • 15. Mai 06

LMU München, Medieninformatik, Instrumented Environments, SS 2006, Andreas Butz 53

  • Huygenic principle (1678): "every

point in a wavefront can be seen as the origin of elementary waves, that propagate with the same velocity and wavelength as the original waves. The enveloping of all elementary waves constitutes the new wavefront."

Wave field synthesis