Magic Wall (working title) David Croft SCOPE Sessions, 18.08.2011 - - PowerPoint PPT Presentation

▶
magic wall
SMART_READER_LITE
LIVE PREVIEW

Magic Wall (working title) David Croft SCOPE Sessions, 18.08.2011 - - PowerPoint PPT Presentation

Jan 30, 2024 213 likes •859 views

Magic Wall (working title) David Croft SCOPE Sessions, 18.08.2011 Magic Wall THE PRINCIPLE Electromagnetic Spectrum Human Eye Model Magic Wall THE HARDWARE Its not a Kinect (Damnit) Has built in IR laser projector Only outline

slide-1
SLIDE 1

Magic Wall

(working title)

David Croft SCOPE Sessions, 18.08.2011

slide-2
SLIDE 2

THE PRINCIPLE

Magic Wall

slide-3
SLIDE 3

Electromagnetic Spectrum

slide-4
SLIDE 4

Human Eye Model

slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9

THE HARDWARE

Magic Wall

slide-10
SLIDE 10

It’s not a Kinect (Damnit)

 Has built in IR laser projector  Only outline needed, not depth  Minimum range 1.2m, maximum range 3.5m  Intrusive  Proprietary connector  640×480 @ 30fps (only)  Most functionality is within the Xbox  Expensive (100€+)

slide-11
SLIDE 11

Inaccurate even at short range

slide-12
SLIDE 12

Camera

 Cheap  Fast frame rate  Good resolution  “Hackable” – to remove IR-blocking filter

and add IR band-pass filter

 Cross-platform drivers

slide-13
SLIDE 13

PlayStation Eye Camera

 640x480 @ 60fps (320x240 @ 120fps)  Free community-written drivers for

Windows, Mac, Linux (from 2.6.29)

 Already in use for open source multi-touch

tables

 IR hacking well documented  Very cheap for its quality (15€)

slide-14
SLIDE 14

Hacking the PS3 Eye

 Get it open (hard!)  Remove built-in IR-blocking filter  Fully-exposed and developed camera film,

  • r floppy disk, work OK as visible light filter

 Better: specialised IR band-pass filter  Get it closed without smashing the CCD

sensor (even harder!)

slide-15
SLIDE 15
slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18
slide-19
SLIDE 19
slide-20
SLIDE 20

IR Bandpass Filter

slide-21
SLIDE 21
slide-22
SLIDE 22
slide-23
SLIDE 23
slide-24
SLIDE 24
slide-25
SLIDE 25
slide-26
SLIDE 26

Infrared Emission

 Filtered incandescent lamps  “IR” heat lamps (but also heat and light, and

expensive)

 Lasers  LEDs

slide-27
SLIDE 27

Infrared floodlight

 Contrast between wall and ambient IR

requires powerful emitters

 Position behind viewers requires unusual

angle

 Should scale well to larger areas  Still looking for a good source

slide-28
SLIDE 28

What wavelength?

 850-875nm seems best

with the cameras I tested

 Also the cheapest  Minor visible red glow,

but only if you look at the LEDs

 Filter will depend on

this too

slide-29
SLIDE 29

850nm

 Bleeds slightly into visible spectrum  Doesn’t seem visible on reflection – but

need to test with a child!

slide-30
SLIDE 30

THE SOFTWARE

Magic Wall

slide-31
SLIDE 31

Requirements

 OpenCV (Open Computer Vision) library  Much useful but cryptic real-time computer

vision functionality

 Processing  Simplified programming environment for

visual artists

 My library  Wraps all of the above.  Will be released soon at www.davidc.net

slide-32
SLIDE 32

Software Flow

slide-33
SLIDE 33

Source

slide-34
SLIDE 34

Source

slide-35
SLIDE 35

Source Controls

slide-36
SLIDE 36

Calibration

slide-37
SLIDE 37

3D Calibration

 Camera and projector are not at the exact

same position

 Lenses and FOV are different anyway  Need to recalibrate so that the final

projected image lines up with bodies

slide-38
SLIDE 38

3D Calibration

 Similar principle to touchscreen

calibration, but in 3D space

 Correlate points in both projector

and camera space

 Naïve implementation ignores

perspective, but is fast and sufficient for not

 OpenCV has calibration routines

slide-39
SLIDE 39

Calibration – Source Image

slide-40
SLIDE 40

Calibration – Control Points

slide-41
SLIDE 41

Calibration - Output

slide-42
SLIDE 42

Calibration Method

 Turn off floodlights  IR LED on a stick  Search for a single blob of a given size range  Take the average of its centre of gravity over

a period of time

 Repeat for other points  Run calibration routine  Then re-project each source frame

slide-43
SLIDE 43

Calibration Markers

slide-44
SLIDE 44

Calibration

slide-45
SLIDE 45

Calibration Controls

slide-46
SLIDE 46

Blur

slide-47
SLIDE 47

Threshold

slide-48
SLIDE 48

Blur and Threshold Controls

slide-49
SLIDE 49

Classification

slide-50
SLIDE 50

Blob Detection

slide-51
SLIDE 51

Blob Filtering

slide-52
SLIDE 52

Contour Approximation

slide-53
SLIDE 53

Blob Detection Controls

slide-54
SLIDE 54

Blob Tracking

slide-55
SLIDE 55

Scene

slide-56
SLIDE 56

Display Controls

slide-57
SLIDE 57

Montage

slide-58
SLIDE 58

Scene Controls

slide-59
SLIDE 59

Example Scenes

 Spotlight  Glow  Image projection  Shadow trail  Flames

slide-60
SLIDE 60

Example Scenes

 Video projection  Insects/flocking  Rain  Beach ball game  Forest

slide-61
SLIDE 61

Example Scenes

 Develop a reference hardware design  Finish and release software as open source  Invite others to develop scenes

slide-62
SLIDE 62

Next Steps

 Find better floodlights  Improve, finish and optimise software  Write more demos  Turn it into a Processing library  Release it as open source software with a

hardware reference design

 Regions of interest  Camera and projector tiling

slide-63
SLIDE 63

Further information

 Documentation will appear over the next

few weeks at www.davidc.net

 david@davidc.net