in Hindustani Music Amruta Vidwans Prateek Verma Preeti Rao - - PowerPoint PPT Presentation

Melodic Style Detection in Hindustani Music

Amruta Vidwans Prateek Verma Preeti Rao

Department of Electrical Engineering Indian Institute of Technology Bombay

Department of Electrical Engineering , IIT Bombay

2 of 25

• f 40

OUTLINE

p

Introduction

n

Objective

p

Style Classification in Indian Classical Vocal Music

n

Literature Survey

n

Previous Work

n

Feature Description

n

Database and Listening Tests

n

Results and Extention to Turkish Music

p

Style Classification In Flute Alap recordings

n

Database and Annotation

n

Signal Characteristics of Discriminatory Ornaments

n

Feature Design

p

Conclusion and Future Work

p

References 2/27

Department of Electrical Engineering , IIT Bombay

3 of 25

• f 40

INTRODUCTION

Objective

p Melodic features to distinguish

n Hindustani, Carnatic and Turkish music n Instrument playing styles

p Basis: Melody line alone suffices for listeners to reliably distinguish musical

styles

p Applications: Extract important metadata automatically

3/27

Department of Electrical Engineering , IIT Bombay

4 of 25

Style Classification in Indian Classical Vocal Music

4/27

Department of Electrical Engineering , IIT Bombay

5 of 25

• f 40

INTRODUCTION

Literature Survey – Similar work done

p Timbral features

n Features based on timbre such as MFCC, delta-MFCC, and spectral features used

by Parul et. al (ISMIR13) to distinguish Indian music genres using Adaboost, GMM etc.

p Timbral + Rhythmic features

n Kini et. al. (NCC10) used these features to do genre classification of North Indian

Classical Music into Quawali, Bhajan, Bollywood etc.

n Liu et. al. (ICASSP09) used timbral, wavelet coefficients and rhythmic features to

classify different styles viz. Arabic, Chinese, Japanese, Indian, Western classical

p Emphasized that the diversity of Indian Classical Music is difficult to model

p Timbral +Melodic features

n Salamon et al (ICASSP12) gave a large number of pitch based features in

addition to timbral features which improved the performance. 5/27

Department of Electrical Engineering , IIT Bombay

6 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC

Pre-processing

p Melody extraction

n Difficulty: Presence of multiple instruments along with voice n Accuracy of the present state of the art automatic pitch trackers ~80% n Semi automatic approach for pitch detection is used to achieve best possible

performance 6/27

Carnatic raga Dwijavanti by artiste R Vedavalli Hindustani raga Jaijaiwanti by artiste Rashid Khan

• riginal

resynthesized

Department of Electrical Engineering , IIT Bombay

7 of 25

• f 40

p Localized contour shape-features

n Previous study used Steady Note (SN) and Gamak Measure (GM) n A steady note: pitch segment of ‘N’ ms with standard deviation less than ‘J’

cents

n Data driven parameters gave highest accuracy (N=400ms J=20cents) n SN: Normalized total duration of steady regions n GM: ratio of number of non steady regions(1sec) having oscillations in

3-7.5Hz to the total number of regions STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC

Feature Description

7/27

Department of Electrical Engineering , IIT Bombay

8 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC

Feature Description

p Primary Shape contour (PS)

n Contour typology proposed by C. Adams [6] for categorizing melodic shapes in 15

types

n Segments taken from silence to silence, assignment done using relation between

Initial, Final, Highest and Lowest pitch values. 8/27

Contour types 12 and 13 in alap section in raga Hindolam by Carnatic artistes (a) T. N. Sheshagopalan (b)M. S. Subalaxmi

(a) (b) 12 13

Department of Electrical Engineering , IIT Bombay

9 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC

Feature Description

p Distance of highest peak in unfolded histogram from Tonic (DistTonic)

n In unfolded histogram the Hindustani alaps are concentrated near the tonic,

Carnatic alap pitch distribution is closer to the upper octave tonic

n Distance of the highest peak from the tonic in the unfolded histogram taken as

feature. 9/27

Rashid Khan raga Todi Sudha Raghunathan raga Subhapanthuvarali

Department of Electrical Engineering , IIT Bombay

10 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC

Feature Description

p Melodic Transitions (MT)

n The overall progression of the melody can be characterized by this n Haar wavelet basis function used to represent the melody n Fifth level approximation is used

p Lower level approximation captures very minute variation whereas higher

level gives a coarse representation.

n Normalized size of upward jumps(>1semitone) in concatenated pitch

contour taken as feature. 10/27

Department of Electrical Engineering , IIT Bombay

11 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC

Database

p Choice of alap section used for labeling a track

n Unmetered section always rendered in the start. n Database Description

p Widely performed raga pairs of same scale interval from Hindustani and Carnatic p Ragas belonging to different scales chosen p Renowned artists of various schools of music chosen

n Total of 120 alap sections equally distributed across both the styles

11/27 Scale Hindustani Raga (No. of clips) Carnatic Raga (No. of clips ) Heptatonic Todi (12) Subhapanthuvarali (14) Pentatonic Malkauns (18) Hindolam (12) Nonatonic Jaijaiwanti (10) Dwijavanthy (14) Octatonic Yaman and Yaman Kalyan (20) Kalyani (20)

Department of Electrical Engineering , IIT Bombay

12 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Listening Tests

p Test Design

n Hypothesis : Melody alone is sufficient to carry out style distinction n Audio clips re-syntheisized with constant timbre sound.

p Removes bias towards artist identity, pronunciation, voice quality. p Volume dynamics retained (Sum of vocal harmonics).

n Interface Description

p User information in terms of training of subject as well as familiarity p Audios divided in 5 sets --12 clips of each Hindustani (H)-Carnatic (C) in each set. p Listening of 10 sec audio mandatory with option of pause, Skipping an audio not allowed p Decision label as H, C or NS (Not Sure) asked for each clip

12/27 Category Accuracy (no of participants) trained <3yrs 74.6% (10) trained 3-10yrs 79.7 % (8) trained >10yrs 89.6 % (2) Amateur 75.2 % (18) Listener 77.5 % (13) Overall Accuracy 76.9 % (51)

Department of Electrical Engineering , IIT Bombay

13 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Automatic classification and feature selection

p Quadratic classifier used on feature sets

n 5 fold CV on 5 different random partitions to avoid any raga bias n Exhaustive search on all possible feature combinations for feature selection n Separability of parameterized distribution taken into account for small dataset

p Distribution of the Log Likelihood ratio (LLR) found for a feature set. p F-ratio computed on distribution LLR as a confidence measure.

n Highest accuracy achieved is 96% for SN, GM, DistTonic, and PS feature set.

13/27

𝑀𝑀𝑆 𝑀𝑀𝑆= ¡ = ¡​ln ln⁠(​𝑀 (​𝑀(​ (​ 𝑦∕ 𝑦∕H )/ )/𝑀(​ (​ 𝑦∕ 𝑦∕C ) ) ) ) 𝐺 ¡− ¡−𝑆𝑏𝑢 𝑆𝑏𝑢𝑗𝑝 𝑗𝑝 ¡= ¡=​(​𝜈 ​𝜈↓𝐼 𝐼 −​𝜈 ​𝜈↓𝐷 ​)↑2 /​𝜏↓ ​𝜏↓𝐼 𝐼 ↑2 +​𝜏↓ ​𝜏↓𝐷 ↑2

Department of Electrical Engineering , IIT Bombay

14 of 25

• f 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Automatic classification and feature selection

p Confusion matrix p Correlation with the listeners

n Subjective test label across all listeners decided by 60% threshold for labeling H,

C or NS

n For objective measure region around 10% of LLR values around 0 taken as NS n Cost value of -1,1 or 0 is assigned according to the agreement between

subjective and objective labels.

n Highest value of correlation was found to be 0.79 across all the feature

combinations 14/27

Obs.

C H NS

True

C 45 12 3 H 8 48 4

(a) (b)

Confusion matrix for (a) listening tests (b) classifier output The horizontal labels correspond to true labels.

Obs.

C H

True

C 58 2 H 3 57

Department of Electrical Engineering , IIT Bombay

15 of 25

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Extention to Turkish Music

15/27

Carnatic raga Dwijavanti by artiste R Vedavalli Turkish makam Nihavent by artiste Hafiz Kemal Bey Hindustani raga Jaijaiwanti by artiste Rashid Khan

Department of Electrical Engineering , IIT Bombay

16 of 25

• f 40

p Experimental Setup and Results

n 60 Taksims considered for classification using the melodic features n The features discussed before were applied to 3 classes. n Highest accuracy of 81.6 % was obtained using SN, GM, MT, DistTonic. n Most confusion occurred for T and C which was validate via subjective

tests

n No category of primary shape was discriminating Turkish-Carnatic clips

Obs. H C T True H 56 3 1 C 1 48 11 T 17 43 Obs. H C T NS True H 131 13 2 4 C 21 118 7 4 T 5 25 114 6 (a) (b)

Confusion matrix for (a) Listening test output (b) Classifier output 16/27

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Extention to Turkish Music

Department of Electrical Engineering , IIT Bombay

17 of 25

Style Classification In Flute Alap recordings

17/27

Department of Electrical Engineering , IIT Bombay

18 of 25

• f 40

INTRODUCTION

Example: Gayaki vs Tantrakari

18/27

Department of Electrical Engineering , IIT Bombay

19 of 25

• f 40

p Gayaki vs Tantrakari styles

n Gayaki : Vocal characteristics incorporated into instrument play.

p Singing voice is fluid, glides used often

n Tantrakari : Instrumental characteristics (particularly plucked string)

p Melodic Leaps eg. Discreteness p Elements synonymous with pluck adapted in Flute : Tonguing p Fast Stroke pattern while playing melody adapted in Flute: Fast Tounging

p Ornament Production

n Glide : Slowly lifting the finger- Rate

controlling the glide shape

n Vibrato : Periodic Pulsations in Air Flow n Fast Tounging : Tongue movement

p Using tongue to articulate notes differently in a melody p Synonymous with Sitar-rapid movements of right hand (Tantrakari)

n Blow Stop :

p Blowing with stops while rendering a note p Periodic blowing patterns (1-2 or 1-1-2) with same/different note

INTRODUCTION

Tantrakari vs Gayaki Style

19/27

Department of Electrical Engineering , IIT Bombay

20 of 25

• f 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Database and Annotation

Artist ¡ Raga ¡ Duration(mins) ¡ Hari Prasad Chaurasia ¡ Jhinjhoti (T-Metadata) ¡ 18:54 ¡ Hansadhwani ¡ 1:50 ¡ Yaman ¡ 2:29 ¡ Sindh Bhairav ¡ 3:30 ¡ Rupak Kulkarni ¡ Hemant (T-Metadata) ¡ 6:50 ¡ Ronu Majumdar ¡ Vibhas ¡ 5:39 ¡ Pannalal Ghosh ¡ Yaman ¡ 2:18 ¡ Shri ¡ 2:20 ¡ Pilu ¡ 1:30 ¡ Nityanand Haldipur ¡ ShuddhaBasant ¡ 2:46 ¡

Four Music Experts unaimously aggred on the two clips as tantrakari

20/27

Department of Electrical Engineering , IIT Bombay

21 of 25

• f 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS Annotation Rules

p Five categories used for annotation

n Tantrakari

p T1 : Blowing same notes with stops p T2 : Steady notes with abrupt movements p T3 : Discreteness with silences p T4 : Discreteness with rapid jumps p T5 : Fast Tounging with any pitch movement

n Gayaki

p G1 : Oscillatory repetitive movements p G2 : Non- Oscillatory repetitive movements p G3 : Glides p G4 : Non-specific continuous pitch p G5 : Vibrato

p

Final characteristic feature taken as the consensus of the feature agreed by all the musicians 21/27

Department of Electrical Engineering , IIT Bombay

22 of 25

• f 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Database and Annotation

Artist ¡ Raga ¡ Duration(mins) ¡ f/T/G(in s) ¡ Hari Prasad Chaurasia ¡ Jhinjhoti (T-Metadata) ¡ 18:54 ¡ 352/465/267 ¡ Hansadhwani ¡ 1:50 ¡ 43/14/33 ¡ Yaman ¡ 2:29 ¡ 57/12/40 ¡ Sindh Bhairav ¡ 3:30 ¡ 50/20/66 ¡ Rupak Kulkarni ¡ Hemant (T-Metadata) ¡ 6:50 ¡ 124/102/111 ¡ Ronu Majumdar ¡ Vibhas ¡ 5:39 ¡ 47/11/44 ¡ Pannalal Ghosh ¡ Yaman ¡ 2:18 ¡ 39/3/66 ¡ Shri ¡ 2:20 ¡ 45/0/29 ¡ Pilu ¡ 1:30 ¡ 133/35/105 ¡ Nityanand Haldipur ¡ ShuddhaBasant ¡ 2:46 ¡ 94/0/41 ¡ q

G: Gayaki T : Tantrakari f: Niether Tantrakari

• r Gayaki

q

Alap-Jod-Jhala composition derived from Sitar. (It mainly contains Tantrakari elements)

q

Two different style artists and their disciple chosen 22/27

Department of Electrical Engineering , IIT Bombay

23 of 25

• f 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Database and Annotation

Artist ¡ Raga ¡ Vibrato ¡ Fast Tounging ¡ Blow - Stop ¡ T3/T4 ¡ G3/G4 ¡ Hari Prasad Chaurasia ¡ Jhinjhoti(T) ¡ P P ¡ P P ¡ P P ¡ P P ¡ P P ¡ Hansadhwani ¡ P ¡ O ¡ P ¡ P ¡ P ¡ Yaman ¡ O ¡ O ¡ O ¡ P ¡ P ¡ Sindh Bhairav ¡ P ¡ O ¡ O ¡ P ¡ P ¡ RupakKulkarni ¡ Hemant(T) ¡ O ¡ P P ¡ P P ¡ P P ¡ P P ¡ RonuMajumdar ¡ Vibhas ¡ P ¡ O ¡ O ¡ P ¡ P ¡ PannalalGhosh ¡ Yaman ¡ P ¡ O ¡ O ¡ P ¡ P ¡ Shri ¡ P ¡ O ¡ O ¡ O ¡ P ¡ Pilu ¡ P ¡ O ¡ O ¡ O ¡ P ¡ NityanandHaldipur ¡ ShuddhaBasant ¡ P ¡ O ¡ O ¡ O ¡ P ¡

23/27 Presence of Tantrakari and Gayaki elements

Department of Electrical Engineering , IIT Bombay

24 of 25

• f 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Production and Acoustic Characteristics

p Blow-Stop & Fast Tonguing – Two of the most distinctive features

n Energy Contour different due to difference in articulation n Higher rate of energy variation due to use of tongue. n Pitch movements do not influence energy movements. n Intensity of Tanpura almost constant

24/27

Department of Electrical Engineering , IIT Bombay

25 of 25

• f 40

p Features capturing acoustic characteristics of Fast Tounging

n Sub-band Peak Ratio (SPR) = n Sub-band FFT Ratio (SFR) =

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Feature Design

• - 2s segments of data across the two styles taken

as data point

• - Ground Truth obtained using manual

annotation

25/27

Classifier Accuracy Quadratic 74.07 Linear 73.15

Department of Electrical Engineering , IIT Bombay

26 of 25

• f 40

p Conclusion

n Successfully proposed melodic features to distinguish vocal music styles n Importance of energy based features brought out for flute style

classification.

n Automatic classification results given with degree of separability

p Future Work

n Extension of database to non-alap recordings in flute style classification

study.

n Addition of new features to distinguish C and T to improve the

classification accuracy 26/27

Department of Electrical Engineering , IIT Bombay

27 of 25

• f 40

1] P. Agarwal, H. Karnick and B. Raj, “A Comparative Study of Indian and Western Music Forms”, ISMIR 2013 2] S. Kini, S. Gulati and P. Rao, “Automatic genre classification of North Indian devotional music” , NCC 2010 3] Y. Liu, Q. Xiang, Y. Wang and L. Cai, “Cultural Style Based Music Classification of Audio Signals,” Acoustics, Speech and Signal Processing, ICASSP, 2009. 4] A. Kruspe et al. “Automatic classification of Music Pieces into Global Cultural Areas ”, In Proc. of 42nd International Conference on Semantic Audio, AES 2011 5] J. Salamon, B. Rocha and E. Gomez, "Musical Genre Classification using Melody Features extracted from polyphonic music signals", International Conference on Acoustics, Speech and Signal Processing, 2012 6] C. Adams, “Melodic Contour Typology” , Ethnomusicology 20.2 : 179-215 (1976) 7] S. Justin, S. Gulati and X. Serra, "A Multipitch Approach to Tonic Identification in Indian Classical Music", ISMIR, 2012 8] C. Clemments, “Pannalal Ghosh and the Bānsurī in the Twentieth Century” , City University of New York, New York City, September 2010

27/27

References