M4 Brno, video processing Pavel Zem k, Stanislav Sumec, Igor Pot - - PowerPoint PPT Presentation

M4 Brno, video processing

Pavel Zemčík, Stanislav Sumec, Igor Potúček, Kteish Abu Ibrahim, Martin Dobsik (currently Helsinki)

• Msc. Students

Goals

• To develop (an automatic) video editing tool
• To provide a video taging tool (currently XML)
• To track and interpret the participants' motion
• To extract features to improve audio processing
• To capture and interpret speaker's gestures
• To create a simple (scaled-down) version of the

audio/video meeting recording equipment

Video editing tool

• Simple „video class“ that allows for simple

software controlled editing of video based eg. on the XML tags – ready and documented (Windows

• nly so far but the interface is portable)
• Research in development of aesthetic metrics for

video to allow automatic editing starts in conjunction with MU Brno and FAVU VUT Brno (the work follows the T. Staudek's Ph.D. work in static image aesthetics published at SIGGRAPH)

• Future extension for camera navigation (no HW)

Video tagging tool

• Video tagging tool available and documented

(currently uses XML directly but can be adopted for a data base)

• Tag definitions
• Tag assignment

by hotkeys

• Tag additional texts
• XML output

Participants' tracking

• Robust identification of the „starting point“ -

colour based; problems – changing camera setup, bad exposure, changing colour statistics

• Tracking of once detected body part – not

finished, need more modelling of body parts motion, need better identification of body parts

• With omnidirectional images we should be finally

able to reliably count paticipants and identify their body parts

Features to improve audio processing

• Simple but most possibly useful features:

identification and position of body parts

(currently we know the position, id manually)

motion of the body parts (centre, bouding rect) mouse/face movement under development, lips

tracking hardly possible because of the resolution (need robust and precise face positioning)

Speakers' gestures

• Models of articulated structures and gesture

interpretation partly done (previous projects)

• The implementation requires robust feature

extraction (not yet done)

• Applications will be the “voting detection and

recognition”, „recording equipment control“, and possibly „emotion detection“

Simplified video/audio capture

• Scaled-down recording equipment built from off-

the-shelf goods (simple and feasible), the motivation also is to capture some „Enlish with accent“ data

• Single camera with hyperbolic mirror probably

suits the needs, at least with HDTV resolution

• The set should consis of a single notebook,

camera (DV or HDTV) with lenses and hyperbolic mirror, and two stereo audio devices (two USB Sound Blasters)

Hyperbolic mirror