Eulerian video magnification: engineering applications of a V1-like - - PowerPoint PPT Presentation

Eulerian video magnification: engineering applications of a V1-like image representation

Bill Freeman Professor and Associate Dept. Head, Electrical Engineering and Computer Science Massachusetts Institute of Technology Joint work with Michael Rubinstein, Neal Wadhwa, Fredo Durand.

Friday, January 3, 14

Steerable pyramid image representation

input

• utput

even and

• dd phase

basis functions

(odd-phase filters shown are from a difgerent figure)

steerable pyramid image representation (only the even phase outputs are shown here)

Friday, January 3, 14

Phase-based Pipeline (SIGGRAPH’13)

Complex steerable pyramid [Simoncelli and Freeman 1995] Temporal filtering on phases

Phase Amplitude

Friday, January 3, 14

4

Friday, January 3, 14

Motion magnification

motion magnified

• riginal

Friday, January 3, 14

Vibration Modes of PVC pipe

Source (20000 FPS) 480Hz (x200) 1200Hz (x400) 2400Hz (x1200) Sequences courtesy of Justin Chen, Civil Engineering, MIT “Piping Vibration Analysis” [Wachel et al. 1990]

Friday, January 3, 14

• riginal

motion magnified (x40, 6 - 11 Hz)

Friday, January 3, 14

still-frames from original and motion magnified (40x, 6 - 11 Hz)

Friday, January 3, 14

I(x,t) = f (x +δ(t))

f (x +δ(t))→ f (Mx + Mδ(t)) f (x +δ(t))→ f (x + Mδ(t)) → f (x +δ1(t)+ Mδ 2(t))

For a translating image A size microscope, with magnification M, gives this: A motion microscope gives this: And we often “focus” the amplification on a particular subband of the temporal signal:

Friday, January 3, 14

• riginal

size magnified by 40 motion magnified by 40

Friday, January 3, 14

Engine photography

Friday, January 3, 14

400 fps video of car engine

Friday, January 3, 14

400 fps video of car engine, 30x motion magnified

Friday, January 3, 14

Original vs motion magnified

Friday, January 3, 14

Frequency (Hz)

500 1000

Power

Source (2 KHz)

Visualization of Vocalization Motions

Friday, January 3, 14

Frequency (Hz)

500 1000

Power

Source (2 KHz) Source (2 KHz) 100 Hz Amplified x100

Fundamental frequency: ~100Hz high-speed video high-speed video, motion magnified x100

Friday, January 3, 14

Frequency (Hz)

500 1000

Power

Source (2 KHz) 220 Hz Amplified x100 Source (2 KHz) 225 Hz Amplified x100

high-speed video high-speed video, motion magnified x100 First harmonic

Friday, January 3, 14

Frequency sweep showing motion components

• f a person at rest.
• riginal

0.1-0.5 Hz amplified 2.5-4.5 Hz amplified 1-2 Hz amplified

Friday, January 3, 14

Eulerian Video Magnification in the wild

Friday, January 3, 14

VideoScope by Quanta Research Cambridge

Friday, January 3, 14

EVM in the Wild: Guinea Pig!

“SuperCreaturefan”: “Guinea pig Tifgany is the first rodent on Earth to undergo Eulerian Video Magnification.” http://www.youtube.com/watch?v=uXOSJvNwtIk Source Motion-magnified

Friday, January 3, 14

EVM in the Wild: Pregnancy

Original Processed

“Tomez85” https://www.youtube.com/watch?v=J1wvFmWv7zY

Friday, January 3, 14

by Erin Knutson (Graphic Design student at Yale)

Friday, January 3, 14

Ways to use the method

• Coding in Matlab

–Matlab code available for last-year’s method

• Upload your own videos

–Upload site for phase-based method.

• Potential collaborations.
• To reach project page, google “Eulerian video

magnification”

24

Friday, January 3, 14

For research project pages, with pointers to code or video upload site:

http://people.csail.mit.edu/mrub/vidmag/ http://people.csail.mit.edu/nwadhwa/phase-video/

Joint work with Michael Rubinstein, Hao-Yu Wu, Eugene Hsu, Neal Wadhwa, .

Friday, January 3, 14