Computational Tools for the Exploration of Melodic Characteristics - PowerPoint PPT Presentation

Computational Tools for the Exploration of Melodic Characteristics CompMusic Seminar, IIT-Madras, Chennai, India 14 Dec, 2013 Sankalp Gulati

Melodic similarity Raga recognition Motif extraction Intonation analysis Melody extraction Tonic identi fj cation

Computation al tools

Computation al tools Tonic

Computation al tools Melody Tonic

Computation al tools Intonation Melody Tonic

Motifs Computation al tools Intonation Melody Tonic

Raga Motifs Computation al tools Intonation Melody Tonic

Similarity Raga Motifs Computation al tools Intonation Melody Tonic

Tonic Identi fj cation

Tonic Identi fj cation 5000 1 f 2 4000 0.8 f 3 3000 0.6 f 4 Tonic 0.4 2000 f 5 f 6 0.2 1000 0 0 Signal processing Learning Accuracy : ~90% !!!

Tonic Identi fj cation 5000 1 f 2 4000 0.8 f 3 3000 0.6 f 4 Tonic 0.4 2000 f 5 f 6 0.2 1000 0 0 Signal processing Learning § Tanpura / drone background sound Accuracy : ~90% !!! § Extent of gamakas on Sa and Pa svara § Vadi, sam-vadi svara of the r ā ga Salamon, J., Gulati, S., & Serra, X. (2012). A multipitch approach to tonic identification in Indian classical music. In Proc. of Int. Conf. on Music Information Retrieval (ISMIR) (pp. 499–504), Porto, Portugal. Bellur, A., Ishwar, V., Serra, X., & Murthy, H. (2012). A knowledge based signal processing approach to tonic identification in Indian classical music. In 2nd CompMusic Workshop (pp. 113–118) Istanbul, Turkey. Ranjani, H. G., Arthi, S., & Sreenivas, T. V. (2011). Carnatic music analysis: Shadja, swara identification and raga verification in Alapana using stochastic models. Applications of Signal Processing to Audio and Acoustics (WASPAA), IEEE Workshop , 29–32, New Paltz, NY.

Tonic Identi fj cation Vignesh Ishwar, Varnam

Tonic Identi fj cation C# Vignesh Ishwar, Varnam

Tonic values in our music collection 150 Number of recordings 100 50 . ¡ . ¡ . ¡ 0 A A# B C C# D D# E F F# G G# A A# B Tonic Frequency

Melody Extraction

§ Pitch (Fundamental frequency-F0) of the lead artist § Pitch estimation § Solo Vs multiple instruments § Melodic contour characteristics § Dual melodic lines in Indian art music

§ Pitch (Fundamental frequency-F0) of the lead artist § Pitch estimation § Solo Vs multiple instruments § Melodic contour characteristics § Dual melodic lines in Indian art music Signal processing Learning Salamon, Justin, and Emilia Gómez. "Melody extraction from polyphonic music signals using pitch contour characteristics." Audio, Speech, and Language Processing, IEEE Transactions on 20.6 (2012): 1759-1770. Rao, Vishweshwara, and Preeti Rao. "Vocal melody extraction in the presence of pitched accompaniment in polyphonic music." Audio, Speech, and Language Processing, IEEE Transactions on 18.8 (2010): 2145-2154. De Cheveigné, A., & Kawahara, H. (2002). YIN, a fundamental frequency estimator for speech and music. The Journal of the Acoustical Society of America, 111, 1917.

Original Audio

Predominant Voice

Pitch of the Predominant Voice

Loudness and Timbral Facets

Intonation Melody Tonic Motifs

Intonation Analysis

Melody Histogram Histogram

Melody Histogram Computation Ga Re Sa Time 1s 2s 1s Salience Svara Sa Re Ga

Tarsos Demo

Intonation Analysis Mohana - G Begada - G

Motif Extraction

Melodic Motifs (Patterns) § Musically repetitive patterns in melody § Various time scales § Gamaka § Pakad § Breath phrase

Motif Extraction and Detection Supervised Detection + ¡ Matching Retrieval Image ¡taken ¡from ¡-­‑ ¡(Mueen ¡& ¡Keogh, ¡2009) ¡

Motif Extraction and Detection Extraction Induction Discovery Un-supervised Image ¡taken ¡from ¡-­‑ ¡(Mueen ¡& ¡Keogh, ¡2009) ¡

Motif Detection 1500 1300 1000 1200 1000 1100 800 1000 600 500 900 400 800 200 700 0 0 600 1500 � Signal processing 1000 500 Pattern detection V. Ishwar, S. Dutta, A. Bellur, and H. Murthy, “Motif spotting in an alapana in Carnatic music,” in Proc. of Int. Conf. on Music Information Retrieval (ISMIR), 2013, pp. 499–504, Curitiba, Brasil. J. C. Ross, T. P. Vinutha, and P. Rao, “Detecting melodic motifs from audio for Hindustani classical music,” in Proc. of Int. Conf. on Music Information Retrieval (ISMIR), 2012, pp. 193– 198, Porto, Portugal.

P1 P1 P1

R ā ga Recognition

Raga Recognition

R ā ga Recognition Yaman Begada Shree Abhogi Behag Charukeshi Darbar Similarity Durga Unknown R ā ga Chordia, P., & Rae, A. (2007, September). Raag Recognition Using Pitch-Class and Pitch-Class Dyad Distributions. In ISMIR (pp. 431-436). Koduri, G. K., Gulati, S., Rao, P., & Serra, X. (2012). R ā ga recognition based on pitch distribution methods. Journal of New Music Research, 41(4), 337-350.

Melodic Similarity

Melodic Similarity

Intonation Similarity

Thank You!!

Introduction

Computation al tools

Tonic Computation al tools

Tonic 2000 Melody Predominant F0 frequency (Cents) 1900 1800 1700 1600 1500 1400 Computation 1300 2.215 2.22 2.225 Time (1 sample = 10 ms) al tools

Tonic 2000 Melody Predominant F0 frequency (Cents) 1900 1800 1700 1600 1500 1400 Computation 1300 2.215 2.22 2.225 Time (1 sample = 10 ms) al tools Intonatio n

Melody Extraction § What is melody? § What is a complete representation for melody? § … § ……

§ What svaras are used in a performance § What is the relative salience of these svaras § What is the intonation of each of each of these svaras § How to represent svara intonation

R ā ga Recognition Yaman Begada Shree Abhogi Behag Charukeshi Darbar Durga Unknown R ā ga

R ā ga Recognition Yaman Begada Shree Abhogi Behag Charukeshi Darbar Durga Unknown R ā ga

R ā ga Recognition Yaman XX Yaman Unknown R ā ga XX Yaman

R ā ga Recognition Yaman XX Yaman Yaman XX Yaman

Tala Images Labels Biographies R ā ga Cover Arts Relations Artist Description Ed. Inst. Info. Listening Tests Ground Truth Evaluation Relevance