Hindustani Classical Music: Methods And Evaluation Strategy Joe - - PowerPoint PPT Presentation

Motif Detection From Audio In Hindustani Classical Music: Methods And Evaluation Strategy

Joe Cheri Ross and Preeti Rao IIT Bombay

Motifs in Hindustani Music

 Melodic motifs or signature phrases are essential building

blocks in Indian Classical music.

 Apart from the swaras that define the raga, it is the

characteristic phrases give it a unique identity [1] Objective of the present work

 Identify all occurrences of melodically similar phrases

in the song given a specific instance of the phrase

Audio example: ‘Jag Mein’ Bandish (Composition)

Rendered by Pt. Ajoy Chakrabarty

Melodic contour extracted by PloyphonicPDA [3]

An Approach to Motif Detection

 Segmentation: find the boundaries (in time) of

candidate phrases. What are the acoustic cues?

 Similarity matching: compute a “melodic

distance” between the given phrase and candidate phrases. What is a good melodic distance measure ?

A Prominent Motif: Mukhda phrase Mukhda is the recurring title phrase of a „Bandish’

(Composition)

Why did we restrict ourselves to Mukhda phrases ?

• The ease of marking ground truth based on lyrical

similarity

• The availability of cues to phrase location from the

rhythmic structure

Mukhda Phrases as seen on the pitch contour

Song: Piya Jag Swaras: D P G P

Segmentation: Characteristic of a Mukhda motif

 Mukhda phrase has a specific location in the rhythmic cycle- around

sam

 Ex: Phrase 'Guru Bina'

 Starts 5 beats before sam (t1)  Ends at sam (t2)

This is the cue for identifying the candidate phrases

Candidate phrase length dependent on the tempo at the instant

Mukhda Phrases on the Pitch Contour

Song: Guru Bina Swaras: S S N R

Performance of Guru Bina by Pt. Ajoy Chakrabarty

Example Identification of ‘Guru Bina’ phrase

Po Positiv ive phr phrases Ne Negative phr phrase Detects phrases melodically similar to „Guru Bina‟ pitch contour Emphatic beat sam Swaras: S S N R

Example : ‘Piya Jag’ Phrases

Po Positiv ive phr phrases Ne Negative phr phrase

Similarity Measures for time series

 Symbolic Aggregate approXimation(SAX) [7]

 Pitch sequence of each phrase is reduced to uniform length(w)  Euclidean distance between phrases is computed

 Dynamic Time Warping(DTW) [6]

 Finds similarity between sequences which vary in time or

speed

 Sakoe-Chiba constraint is enabled to avoid any pathological

warping

1.

Extract candidate phrases(same rhythmic structure) from the song(pitch contour) by automatic detection of the sam (or similar bols)

2.

With the help of annotated ground truth, find the positive phrases among the generated

3.

Compare each positive candidate phrase with the all phrases using similarity measures

Experiment

To evaluate the performance of similarity measures

• The location of positive phrases is manually annotated in the song.
• The pitch sequence of the song (pitch value for each 10ms)

Experiments were done with quantized and un-quantized pitch

Dataset

Expt Bandish Singer #Phrases POS NEG A Guru Bina

• Pt. Bhimsen Joshi

156 715 B Guru Bina Ajoy Chakraborty 1056 9735 C Jana na na na Pt. Bhimsen Joshi 272 1649 D Piya Jaag Kishori Amonkar 1892 7744 E Guru Bina BJ vs AC 429 3835

'Piya Jaag' Distance Distribution

ROC of DTW and SAX Song: ‘Piya Jaag’

(This work has been reported in Proc. ISMIR 2012 )

Hit rate- 87% False Alarm- 3.2 %

Why it is Challenging ?

 Melodically similar motifs may not occur at the same

location in the rhythmic cycle.

 Make it difficult to identify right candidate phrases to be

compared with

 Results in increase in number of candidate phrases, thus the

complexity

Extension to other phrases

Mukhda phrase: ‘Jag Mein Kachu’

Emphatic beat sam Location of Mukhda phrases is consistent w.r.t to location of emphatic beat sam in rhythmic cycle Swaras: G-R-SNRS-N-D-N-S N-NDS

Non-Mukhda phrase N-D-S

• N-D-S is one of the prominent phrases in this bandish
• Location of phrases are not consistent in the rhythmic cycle
• Range of variations due to improvisations is high compared to Mukhda phrases.

Vistar(Variations) of the phrase N-D-S

• All these phrases are to be identified as similar motifs
• Phrase ending in Nyas swar(long note) S.

Long note S

Approaches

1.

Identify motifs based on repeating patterns

2.

Identify motifs based on potential segment boundary cues and cluster

Approach 1:

 Symbolic sequence is derived from the pitch contour  Crochemore algorithm[4,5] extracts repeating patterns

from the input symbolic sequence.

 Complexity of algorithm- O(n log n)

 n- length of sequence

Find repeating patterns from the symbolic sequence and similar patterns are grouped together.

Approach 1: Crochemore Algorithm

 Crochemore algorithm extracts repeating patterns from

symbolic sequence.

 Example:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

{1,4,9,11,18}S S R G S R G P G S R S R G P G P G S {2,5,10,12}R {3,6,8,13,15,17}G {7,14,16}P {1,4,9,11}SR {2,5,12}RG {10}RS {3,8,17}GS {6,8,13,15}GP {1,4,11}SRG {9}SRS {3,8}GSR {6,13,15}GPG {1}SRGS {4,11}SRGP {3}GSRG {8}GSRS {6,15}GPGS {13}GPGP {4,11}SRGPG {6}GPGSR

Approach 1: Experiment Method

• Annotation of location of motifs and the belonging cluster.
• Symbolic sequence from the pitch contour

1.

Crochemore algorithm can get the motifs at different levels from the symbolic sequence

2.

Remove short length motifs

3.

With the help of annotated ground truth, find the purity and rand index of clustering

Approach 2:

• 1. Pauses(Silence) occurs at major boundaries (lyrical

phrase boundaries)

• 2. Nyasa(Long notes) occurs at most of the boundaries
• 3. Recurring patterns

Cues to Segmentation:

Find motif boundaries with segmentation cues and cluster similar motifs

Approach 2: Experiment Method

1.

Extract candidate phrases by segmentation from the song(pitch contour)

2.

Find similar motifs using similarity measures and cluster(Agglomerative) them

3.

With the help of annotated ground truth, find the purity and rand index of clustering

• Annotation of the location of motifs and the belonging cluster.
• The pitch sequence of the song (pitch value for each 10ms)

Conclusion & Future Work

 Detecting phrase motifs is challenging due to the inherent

• variability. However:

 Prominent swaras remains the same (Ex: N D S)  Explicit phrase segmentation cues need to be further explored

 Time-series pattern matching methods may be extended

to motif discovery (i.e. no prior knowledge about motifs is available)

References

[1] J. Chakravorty, B. Mukherjee and A. K. Datta: “Some Studies in Machine Recognition

• f Ragas in Indian Classical Music,” Journal of the Acoust. Soc. India, Vol. 17, No.3&4,

1989. [2] S. Rao, W. van der Meer and J. Harvey: “The Raga Guide: A Survey of 74 Hindustani Ragas,” Nimbus Records with the Rotterdam Conservatory of Music, 1999. [3] V. Rao and P. Rao: “Vocal Melody Extraction in the Presence of Pitched Accompaniment in Polyphonic Music,” IEEE

• Trans. Audio Speech and Language

Processing,

• Vol. 18, No.8, 2010.

[4] M. Crochemore: “An Optimal Algorithm for Computing the Repetitions in a Word,” Information Processing Letters, Vol.12, No.5, 1981.

[5] E. Cambouropoulos: “Musical parallelism and melodic segmentation: A computational approach,” Music Perception: An Interdisciplinary Journal, Vol.23, No.3, 2006 [6] D. Berndt and J. Clifford: “Using Dynamic Time Warping to Find Patterns in Time Series,” AAAI-94 Workshop on Knowledge Discovery in Databases, 1994. [7] J. Lin, E. Keogh, S. Lonardi and B. Chiu: “A Symbolic Representation of Time Series, with Implications for Streaming Algorithms,” In Proc. of the Eighth ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery, 2003. [8] A. Mueen , E. Keogh , Q. Zhu and S. Cash: “Exact Discovery of Time Series Motifs,”

• Proc. of the SIAM International Conference on Data Mining, 2009.

[9] J. Ross, T.P. Vinutha and P.Rao: “Detecting Melodic Motifs From Audio For Hindustani Classical Music,” Proc. of Int. Soc. for Music Information Retrieval Conf. (ISMIR), 2012.