[PPT] - Multimedia Data Processing on CIEL Arman Idani 14 Feb 2012 R202 PowerPoint Presentation

SLIDE 1

Multimedia Data Processing on CIEL

Arman Idani 14 Feb 2012 R202 – Data Centric Networking

SLIDE 2

Machine Learning on DC

Apache Mahout (library for Hadoop)
Tons of independent codes
Only on textual/graph content
No multimedia input
Speech, music, image, video

SLIDE 3

Goal

Developing an extension for CIEL
Prepare multimedia data for processing
Tasks should be heavily parallelized
Each task should be able to spawn new tasks (iteration)

SLIDE 4

Model-based Multimedia ML

𝑦1 𝑦2 ⋮ 𝑦n

Labels

𝑧1

y2

⋮ yn

Feature Extraction Training Model

Recognition

𝑄(𝑏) 𝑄(𝑐) 𝑄(𝑑) ⋮ 𝑄(𝑜)

SLIDE 5

Dataset

Multimedia input + time-stamped labels

352150000 416514000 Db 416514000 449336000 Ab 449336000 511888000 Ab 511888000 543612000 Gb 543612000 575153000 Db 575153000 639137000 Ab 639137000 670810000 Gb 670810000 701669000 Db 701669000 717149000 Db 717149000 732513000 Bb:min 732513000 764054000 F:min 764054000 796062000 Eb:min7

SLIDE 6

Challenge?

Terabytes of data
Very difficult training
Solution?
Group each label together

SLIDE 7

Solution

352150000 416514000 Db 416514000 449336000 Ab 449336000 511888000 Ab 511888000 543612000 Gb 543612000 575153000 Db 575153000 639137000 Ab 639137000 670810000 Gb 670810000 701669000 Db 701669000 717149000 Db 717149000 732513000 Bb:min 732513000 764054000 F:min 764054000 796062000 Eb:min7

352150000 416514000 Db 543612000 575153000 Db 670810000 701669000 Db 701669000 717149000 Db 416514000 449336000 Ab 449336000 511888000 Ab 575153000 639137000 Ab

SLIDE 8

Not that Simple!

Terabytes of data
or gigabytes of labels!
Image and Video sources
No longer can cut audio
Need to extract objects from visual sources
Huge in interactive ads

SLIDE 9

Extractor

def extractor(

string <source_path>, string <transcription_path>, boolean <spawn> *, string <intermediate_path> )

SLIDE 10

System Overview

Input 0 Input n Input 2 Input 3 … Input n-1 Input 1

worker worker worker

Extractors

Int 0 Int 2 Int 3 Int 1 Int 7 Int 9 Int 3 Int 2 Int 0 Int 1 Int 3 Int 4

worker worker worker

Reducers

Data Label 0 Data Label m Data Label 2 Data Label 3 … Data Label m-1 Data Label 1

SLIDE 11

System Overview (cont.)

Data Label 0 Data Label m Data Label 2 Data Label 3 … Data Label m-1 Data Label 1

worker worker worker

Data Analyser Feature 0 Feature t Feature 2 Feature 3 … Feature t-1 Feature 1

SLIDE 12

Data Analyser

def analyser (

string <source_path>, boolean <spawn> *, string <destination_path> )

SLIDE 13

Overview

Developer only provides “extractor”, “analyser” and a config
CIEL (and the extension) takes care of the rest
Built-in support for audio and video “reducers”
Sample project for audio processing (MFCC) will be developed

for evaluation

SLIDE 14