Multimodal Music Tagging Task Nicola Orio University of Padova - - PowerPoint PPT Presentation

multimodal music tagging task
SMART_READER_LITE
LIVE PREVIEW

Multimodal Music Tagging Task Nicola Orio University of Padova - - PowerPoint PPT Presentation

Multimodal Music Tagging Task Nicola Orio University of Padova Cynthia C.S. Liem Delft University of Technology Geoffroy Peeters IRCAM Markus Schedl Johannes Kepler University 1 CLEF2012, Rome 18/09/2012


slide-1
SLIDE 1

Nicola Orio – University of Padova Cynthia C.S. Liem – Delft University of Technology Geoffroy Peeters – IRCAM Markus Schedl – Johannes Kepler University

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

1

Multimodal Music Tagging Task

slide-2
SLIDE 2

Evaluation Initiatives – 1

  • MIREX (since 2004)
  • Community-based selection of tasks
  • Many tasks address audio feature extraction algorithms
  • Participants submit algorithms that are run by organizers
  • Music files are not shared with participants
  • Million Song Dataset (since 2011)
  • One task on music proposed by organizers
  • Participants can run their
  • Audio features are computed using proprietary algorithms
  • Only features are shared with participants

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

2

slide-3
SLIDE 3

Evaluation Initiatives – 2

  • Quaero-Eval (since 2012)
  • Tasks agreed with participants
  • Strategies to grant public access to evaluation results
  • Participants run training experiments on a shared repository
  • Runs on test set made by the organizers
  • MediaEval (since 2008)
  • Started as VideoCLEF
  • Scope extended to other media
  • Emphasis on multimodality
  • Regular tasks, plus Brave New Tasks (e.g. MusiClef)

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

3

slide-4
SLIDE 4

Goals of MusiClef

  • To focus evaluation on professional application scenarios
  • Textual description of music items
  • To grant replication of experiments and results
  • Feature extraction phase is crucial
  • To promote the exploitation of multimodal sources of

information

  • Content (audio) + Context (tags & webpages)
  • To disseminate music related initiatives
  • Outside the music information retrieval community

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

4

slide-5
SLIDE 5

The workflow

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

5

MusicCLEF

not public

MusicCLEF

public web site

Participant

participant pc .

last.fm

webservice MP3 Library Low level features

if(conn SELEC WHERE print

features extractor

1.read

<script var a= var xl if(xls

Participant algorithms Low level features tags

if(conn SELEC WHERE print

last.fm data

3 . d

  • w

n l

  • a

d

  • 4. read

results Results evaluation Campaign Participant Results

if(conn SELEC WHERE print

last.fm data

  • 4. read
  • 5. produce
  • 2. produce

1 . h t t p r e q u e s t

  • 7. publish

6 . s u b m i t

slide-6
SLIDE 6

Multimodal Music Tagging

  • Definition
  • Songs of a commercial music library need to be categorized

according to their usage in TV and radio broadcasts (e.g. soundtracks, jingles)

  • Practical motivation
  • The search for suitable music for video productions is a

major activity for music professionals

  • Collaborative filtering systems are taking their role
  • Notwithstanding their known limitations: long-tail, cold start…
  • Annotating professional music libraries is another important

professional activity

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

6

slide-7
SLIDE 7

Human Assessment

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

7 Different sources of information are routinely exploited by professionals to overcome limitations of individual media

slide-8
SLIDE 8

Test Collection – 1

  • Individual songs of pop and rock music
  • 1355 songs (form 218 artists)
  • Social tags
  • Gathered from Last.fm API
  • Multilingual sets of Web pages related to artists+albums
  • Mined querying Google
  • Acoustic features: MFCC (using MIRToolbox) with a

window length of 200ms and 50% overlap

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

8

slide-9
SLIDE 9

Test Collection – 2

  • Test collection created starting from the “500 Greatest

Songs of All Time” (Rolling Stone)

  • Expected high number of social tags and web pages
  • Ground truth created by experts in the domain
  • 355 tags selected (167 genre, 288 usage)
  • Tags associated to less than 20 songs were discarded
  • Reference implementation in Matlab
  • Participants has an example to run a complete experiment
  • Code for the evaluation made already available

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

9

slide-10
SLIDE 10

Conclusions

  • Participants are still submitting their runs
  • So far, 8-10 perspective participants have been involved
  • We are collecting the runs
  • Additional evaluation will be run on subsets of tags
  • Grouping tags in classes related to:
  • Affective, situational, sociocultural aspects
  • Correlate different modalities with different aspects
  • Results will be presented at MediaEval Workshop
  • Pisa, 4-5 October 2012

CLEF2012, Rome 18/09/2012 MusiClef: Multimodal Msic Tagging Task

10