A Survey on Music Retrieval Systems Using Microphone Input Ladislav - - PowerPoint PPT Presentation

▶
a survey on music retrieval systems using microphone input
SMART_READER_LITE
LIVE PREVIEW

A Survey on Music Retrieval Systems Using Microphone Input Ladislav - - PowerPoint PPT Presentation

Feb 23, 2024 200 likes •512 views

A Survey on Music Retrieval Systems Using Microphone Input Ladislav Mark 1 , Jaroslav Pokorn 1 , Martin Ilk 2 1 Charles University, Prague 2 Vienna University of Technology, Vienna Music Information Retrieval (MIR) It has many

slide-1
SLIDE 1

A Survey on Music Retrieval Systems Using Microphone Input

Ladislav Maršík1, Jaroslav Pokorný1, Martin Ilčík2

1 Charles University, Prague 2 Vienna University of Technology, Vienna

slide-2
SLIDE 2

Music Information Retrieval (MIR)

slide-3
SLIDE 3

It has many applications

slide-4
SLIDE 4

Motivation

  • Understand recent MIR systems
  • Find out where we can make improvements

– Recognizing – Segmenting – Annotating – Recommending – Retrieving – Composing – Notation – Storing – Playback – Understanding

slide-5
SLIDE 5

Music Retrieval

  • 1. Audio Fingerprinting
  • 2. Whistling and Humming Queries
  • 3. Cover Song Identification

1. 2. 3.

slide-6
SLIDE 6

Audio Fingerprinting

INPUT: Song recording

slide-7
SLIDE 7

Audio Fingerprinting

INPUT: Song recording OUTPUT: The exact match

slide-8
SLIDE 8

Audio Fingerprinting Wang and Smith: An Industrial-Strength Audio Search Algorithm (2002) “Combinatorially hashed time-frequency constellation analysis”

Time-Frequency Constellation analysis Combinatorially hashed

slide-9
SLIDE 9

Audio Fingerprinting Wang and Smith: An Industrial-Strength Audio Search Algorithm (2002)

Time-Frequency spectrogram

slide-10
SLIDE 10

Audio Fingerprinting Wang and Smith: An Industrial-Strength Audio Search Algorithm (2002)

Constellation analysis

slide-11
SLIDE 11

Audio Fingerprinting Wang and Smith: An Industrial-Strength Audio Search Algorithm (2002)

Constellation analysis

slide-12
SLIDE 12

Audio Fingerprinting Wang and Smith: An Industrial-Strength Audio Search Algorithm (2002)

Combinatorially hashed

h(f1,f2,t2-t1) | t1

slide-13
SLIDE 13

Audio Fingerprinting Wang and Smith: An Industrial-Strength Audio Search Algorithm (2002)

Summary

  • Short search time: 5-500 milliseconds / query
  • Robust to noisy environment
  • Possible extension to abstract from tonality
  • Only exact match results
slide-14
SLIDE 14

Audio Fingerprinting State-of-the-art

  • No benchmarking until recently

(focus on commercial deployment)

slide-15
SLIDE 15

Audio Fingerprinting State-of-the-art

  • No benchmarking until recently

(focus on commercial deployment)

  • Various indexing techniques

and peeks comparison algorithms

slide-16
SLIDE 16

Audio Fingerprinting State-of-the-art

Yang (2001)

Peek sequence: P1P2P3 …

Euclidean distance

slide-17
SLIDE 17

Audio Fingerprinting State-of-the-art

  • No benchmarking until recently

(focus on commercial deployment)

  • Various indexing techniques

and peeks comparison algorithms

  • New use cases: Advertisement, TV program
slide-18
SLIDE 18

Whistling and Humming Queries

INPUT: Whistling or Humming

slide-19
SLIDE 19

Whistling and Humming Queries

INPUT: Whistling or Humming OUTPUT: Song containing the melody

slide-20
SLIDE 20

Whistling and Humming Queries Shen and Lee: Whistle for Music (2007)

  • Whistle: 700Hz-2.8KHz
  • Translation to MIDI (Query and DB)
  • String matching methods
slide-21
SLIDE 21

Whistling and Humming Queries Shen and Lee: Whistle for Music (2007)

slide-22
SLIDE 22

Whistling and Humming Queries Unal et al.: Query by Humming Systems (2008)

  • Use of fingerprinting (relative pitch movement)
slide-23
SLIDE 23

Whistling and Humming Queries State-of-the-art

Benchmarking: MIREX 2014

(Music Information Retrieval Evaluation Exchange) http://www.music-ir.org/mirex/wiki/MIREX HOME

  • Hou et al.: Hierarchical K-means tree, dynamic progr.
  • MusicRadar
slide-24
SLIDE 24

Cover Song Identification

INPUT: Song / Recording OUTPUT: Cover song / Performances

slide-25
SLIDE 25

Cover Song Identification Khadkevich and Omologo: CSI Using Chord Profiles (2013)

slide-26
SLIDE 26

Cover Song Identification Kim et al.: Music Fingerprint Extraction Use of Covariance Matrix Fingerprint, Beat synchronization

slide-27
SLIDE 27

Cover Song Identification State-of-the-art

Benchmarking: MIREX 2014

(Music Information Retrieval Evaluation Exchange) http://www.music-ir.org/mirex/wiki/MIREX HOME

  • Academia Sinica (Tsai, Wang): Melody extraction
  • Bordeaux: Local alignment of chroma sequences

Overall 80-90% precision of identifying covers

slide-28
SLIDE 28

Proposals for improvements

  • Low-level vs. High-level techniques
  • Melody, Harmony, Tonality, Rhythm, Tempo
  • Stabilize descriptors and use DTW to find

similarities

  • Combine Cover Song Identification with

Microphone input methods

slide-29
SLIDE 29

Summary

Survey on Music Retrieval Systems:

  • Audio Fingerprinting
  • Whistle and Humming Queries
  • Cover Song Identification
  • Proposal for improvements
slide-30
SLIDE 30

Thank you for your attention