Carnatic Music: A Computational Perspective Hema A Murthy - - PowerPoint PPT Presentation

▶
carnatic music a computational perspective
SMART_READER_LITE
LIVE PREVIEW

Carnatic Music: A Computational Perspective Hema A Murthy - - PowerPoint PPT Presentation

Oct 06, 2022 107 likes •422 views

Carnatic Music: A Computational Perspective Hema A Murthy Department of Computer Science and Engineering IIT Madras hema@cse.iitm.ac.in e-mail: hema@cse.iitm.ac.in December 13 2013 MIR Indian Music Preliminaries Tonic Gamak a s in

slide-1
SLIDE 1

Carnatic Music: A Computational Perspective

Hema A Murthy Department of Computer Science and Engineering IIT Madras hema@cse.iitm.ac.in

e-mail: hema@cse.iitm.ac.in

December 13 2013

slide-2
SLIDE 2

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Outline MIR and Carnatic Music Carnatic Music Concert Preliminaries Tonic Melodic cues Processing the Drone Gamak¯ as in Carnatic Music Cent filterbanks Mridangam stroke transcription Modes of the mridangam Transcription Pitch Extraction Conclusions

Carnatic Music: A Computational Perspective

slide-3
SLIDE 3

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Music Information Retrieval and Carnatic Music

  • Indian classical music – rich repertoires, many traditions, many genres
  • An oral tradition
  • Well established teaching and learning practices
  • Hardly archived and studied scientifically
  • Indian classical music is rich in Manodharma – improvisation
  • Difficult to analyse and represent using ideas from Western Music
  • Objective: Enhance experience through MIR

Carnatic Music: A Computational Perspective

slide-4
SLIDE 4

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Structure of a Carnatic Music Concert: A listener’s perspective

  • A concert is made up a sequence of items.
  • Items correspond to different forms.
  • Each form has a specific characteristic.
  • R¯

agas are seldom repeated (except in thematic concerts)

  • One or more pieces in a concert are taken up for elaboration
  • Kriti:
  • an alaapana, a composition (pallavi, anupallavi, charanam), niraval, svaraprasthara, solo

percussion

  • Raagam Taanam Pallavi (RTP):
  • RTP: an alaapana, a taanam, a composition (pallavi only), svaraprasthara, solo

percussion.

  • Pallavi is rendered at different speeds with niraval.
  • Svaraprasthara may include multiple melodies.
  • Rhythmic cycles chosen – complex (e.g. Adi taalam (tisra nadai))
  • Other types: padham, jaavali, viruttam, slokam, varnam,

The Main item and RTP are generally the hallmarks of a concert.

Carnatic Music: A Computational Perspective

slide-5
SLIDE 5

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Primary Aspects of Indian Classical Music from a Computational Perspective I

  • Tonic – the base sur chosen by the musician to render the music.
  • Each musician has his/her own tonic
  • Bombay Jayashree: 220 Hz, T M Krishna: 140 Hz, MDR: 110 Hz.
  • R¯

aga or melody

  • Gamak¯

as – the inflection of notes

  • Gamak¯

as are associated with phrases of r¯ agas1

  • Exploration of ¯

Al¯ apana through the relevant phrases

  • Phrases are rendered in different tempos
  • Phrases are derived from compositions – especially from the trinity
  • Talas
  • Strong tradition of rhythm
  • Carnatic Music – kalai, nadai, jaati.
  • Mrudangam stroke analysis
  • Segmentation of tani
  • Can we transcribe the same?
  • Other Aspects 1: The concert is more like a conversation between the artist and

the audience.

Carnatic Music: A Computational Perspective

slide-6
SLIDE 6

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Primary Aspects of Indian Classical Music from a Computational Perspective II

  • The appreciation by the audience is more a “here and now”2
  • A concert is replete with applauses – can we use these to segment and archive them

in terms of pieces?

  • Other Aspects 2: Since motivic analysis is based on pitch – and most algorithms

fall short – explore new algorithms for pitch based on phase.

1T M Krishna and Vignesh Ishwar, “Carnatic Music: Svara, Gamaka, Motif and Raga Identity”, 2nd CompMusic Workshop, Istanbul, Turkey 2M V N Murthy, “Applause and Aesthetic Experience,”http://compmusic.upf.edu/zh-hans/node/151

Carnatic Music: A Computational Perspective

slide-7
SLIDE 7

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Tonic I

3.

  • Tonic – A fundamental concept of Indian classical music
  • Tonic – Pitch chosen by the performer to serve as reference
  • The svara Sa in the middle octave range is the tonic
  • Drone is played to establish tonic – Tanpura/Tambura
  • Accompanying instruments also tune to the tonic
  • Melodies defined relative to tonic

Carnatic Music: A Computational Perspective

slide-8
SLIDE 8

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Tonic II

1000 2000 3000 4000 5000 6000 40 80 120 160 200 240 280 320 Frames Frequency Hz 1000 2000 3000 4000 5000 6000 40 80 120 160 200 240 280 320 Frames Frequency Hz

Tonic 1 Tonic 2 Drone 1 Drone 2 Alapana

3Ashwin Bellur, “Automatic identification of tonic in Indian classical music,” MS Thesis, IIT Madras, 2013

Carnatic Music: A Computational Perspective

slide-9
SLIDE 9

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Approaches to identify tonic

  • Music provides various cues to the listener about identity of the tonic
  • Cues can be divided into two broad classes
  • 1. Melodic characteristics of the music 1
  • 2. Tuning of the drone2

1S Arthi H G Ranjani and T V Sreenivas. Shadja, swara identification and raga verification in alapana using stochastic models. WASPAA 2011

pages 29-32, 2011.

2Salamon, J., S. Gulati, and X. Serra (2012). A multipitch approach to tonic identification in indian classical music. In Proc. of ISMIR, 157163

Carnatic Music: A Computational Perspective

slide-10
SLIDE 10

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Cue 1 - Melodic characteristics of the music

  • Mono pitch extracted from the audio – prominent pitch
  • Pitch range: 40 to 700 Hz, window size = 133ms
  • Histograms as primary representation
  • Typical Histograms for Hindustani and Carnatic music (bin width 1 Hz)

50 100 150 200 250 300 350 400 1000 2000 3000 4000 5000 6000 Frequency (Hz) Number of Instances (Carnatic Item)

Figure: Carnatic Pitch Histogram

50 100 150 200 250 300 350 400 1000 2000 3000 4000 Frequency (Hz) Number of Instances (Hindustani Item)

(b)

Figure: Hindustani Pitch Histogram

Carnatic Music: A Computational Perspective

slide-11
SLIDE 11

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Pitch Histograms based processing

  • 1. A peak indicating note Sa always present, not necessarily the tallest peak
  • 2. Drone and percussion ensure a peak at Sa
  • 3. Histogram envelope is almost continuous due to gamakas. ( Example

)

  • 4. Fixed ratio between peaks representing svara Sa and Pa
  • 5. Less inflected nature of Sa and Pa
  • 6. Characteristics more prominent in Carnatic music

Carnatic Music: A Computational Perspective

slide-12
SLIDE 12

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Cue 2 - Drone

  • Determine tuning of the drone to identify tonic
  • Drone omnipresent in Indian classical music
  • Strings of the tambura tuned to indicate svara Sa
  • Tuning of one of the strings varies depending on the raga being performed
  • Attempt to develop fast tonic identification techniques with minimal data

Figure: Spectogram of an excerpt of Carnatic music

Example 1 Example 2 Example 3

Carnatic Music: A Computational Perspective

slide-13
SLIDE 13

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Determining tuning of the drone

  • Drone omnipresent in the background
  • Drone extracted in low energy regions
  • Lead vocal frequencies predominantly occupy middle and upper octaves
  • Drone frequently registers pitch values at the lower octave Sadja/Sa

500 1000 1500 2000 2500 3000 3500 4000 100 200 300 Frame Number

Frequency Hz

50 100 150 200 250 300 0.5 1

Frequency Hz

(b) (a)

panchama lower sadja middle sadja

Carnatic Music: A Computational Perspective

slide-14
SLIDE 14

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Drone Prominent Frames

  • Pick frames with drone as the prominent source
  • A host of low level audio descriptors were employed
  • Pitch estimated using a selected bag of frames using signal processing,

dictionary learning methods (Non Negative Matrix Factorisation (NMF)). Results:

  • Performance almost as high as 90% on 1.5min of data when signal processing

cue is used.

  • Performance 98% with drone on 1.5mins of data.

Carnatic Music: A Computational Perspective

slide-15
SLIDE 15

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Gamak¯ as in Carnatic Music4

  • Written forms of a r¯

aga

  • ArOhana – sequence of notes in ascending order (generally starts from the tonic Sa).
  • AvarOhana – sequence of notes in descending order (generally starts from the tonic

˙ Sa).

  • Vocal/Instrumental form of a r¯

aga

  • Replete with Gamak¯

as

  • Although specific placeholders – notes can meander quite signficantly about the

placeholder.

  • The “note uttered” and “note sung” need not have a one-to-one correspondence

4Vignesh Ishwar, Shrey Dutta, Ashwin Bellur and Hema A Murthy, “Motif spotting in an Alapana in Carnatic music,” ISMIR 2013.

Carnatic Music: A Computational Perspective

slide-16
SLIDE 16

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Classification of Gamak¯ as (SSP – Subbarama Dikshitar, Dasavidha Gamak¯ a –as told to us by the experts)

  • A finite number of meandering patterns are defined
  • kampitam, jaaru, vali, spuritham, ...
  • The Gamak¯

as are stitched together/one gamak¯ a is overlayed on the other

  • Other than gamak¯

as, “brikhas” are also used.

  • Standard set of melodic phrases are observed in every piece.

Carnatic Music: A Computational Perspective

slide-17
SLIDE 17

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Characteristic Pitch and Spectrogram for some of the Gamak¯ as Some example Gamak¯ as created by Vignesh Ishwar – tOdi Kampitham: Jaaru: Odukkal: Nokku: Orikkai:

Carnatic Music: A Computational Perspective

slide-18
SLIDE 18

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Gamak¯ as in a given ¯ al¯ apana Vignesh tOdi ¯ Al¯ apana: KVN bhairavi ¯ Al¯ apana:

Carnatic Music: A Computational Perspective

slide-19
SLIDE 19

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Summary of pitch analysis on complete ¯ al¯ apanas

  • The musician does not seem to always stick to the grammar of the gamak¯

a.

  • Some definite phrases do exist
  • Most keen listeners identify r¯

agas without difficulty.

  • A set of Time-Frequency (T-F) motifs?
  • T-F representation of pitch?
  • Mapping from r¯

aga to the language as defined by musicians required.

  • Derive motifs for r¯

agas using “machine learning?”

Carnatic Music: A Computational Perspective

slide-20
SLIDE 20

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Histograms of notes for different r¯ agas I

−1500 −1000 −500 500 1000 1500 2000 2500 500 1000 1500 Shankarabharana Pitch Histogram Frequency in Cents −1500 −1000 −500 500 1000 1500 2000 2500 1000 2000 3000 4000 Kalyani Pitch Histogram Frequency in Cents Sa R2 G3 M1 P D2 N3 Sa Sa R2 G3 M2 P D2 N3 Sa

Figure: Pitch Histograms of Ragas Kalyani and Sankaraabharana

Carnatic Music: A Computational Perspective

slide-21
SLIDE 21

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Histograms of notes for different r¯ agas II

  • Significant band of frequencies around every peak that is also frequent.
  • The two r¯

agas have only one note that is different – but their histograms are very different.

  • Pitch contour is seamless and continuous.

Carnatic Music: A Computational Perspective

slide-22
SLIDE 22

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Motifs in a r¯ aga

1 2 3 4 −500 500 1000 1500 2000 Time in Minutes Frequency in Cents

Pitch Contour of an Alapana

0.5 1 600 800 1000 1200 1400 Time in Seconds Frequency in Cents

Motif1

0.5 1 600 800 1000 1200 1400 Time in Seconds Frequency in Cents

Motif2

0.5 1 600 700 800 900 1000 1100 1200 1300 Time in Seconds Frequency in Cents

Motif3

Figure: a) Motifs Interspersed in an Alapana; b) Magnified Motif

  • The motif is repeated in different parts of the ¯

Al¯ apana

  • There are a number of motifs for each r¯

aga

  • Question: Motif as a query, can we locate it in an ¯

Al¯ apana?

  • ¯

Al¯ apana: Long, erroneous pitch contour, computationally intensive. Saddle points in tandem with Rough LCS used to locate motifs. Shrey Dutta will discuss this work.

Carnatic Music: A Computational Perspective

slide-23
SLIDE 23

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

A new feature for analysis of Indian Music I Tonic shift – causes a linear shift in cent scale. A possible solution: cent filterbank banks

  • Analysis of a concert in Indian Music depends on tonic.
  • Motifs can span more than one octave.
  • Cochlea – can be modeled by a bank of constant Q filters.
  • For Indian music: Normalise by tonic – place filters uniformly on the cent scale:

Cent Filterbanks

Carnatic Music: A Computational Perspective

slide-24
SLIDE 24

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Cent Filterbanks

  • 5

5 10 15 20 25 30 35 40

  • 20

20 40 60 80 100 120 140 12 14 16 18 20 22 24 26 28 30

  • 5

5 10 15 20 25 30 35 40

  • 20 0 20 40 60 80 100

120 140 160 180 14 16 18 20 22 24 26 28 30 32 34

  • 5

5 10 15 20 25 30 35 40

  • 20 0 20 40 60 80

100 120 140 160 180 200 14 16 18 20 22 24 26 28 30

  • 5

5 10 15 20 25 30 35 40

  • 20

20 40 60 80 100 120 12 14 16 18 20 22 24 26 28 30

  • 5

5 10 15 20 25 30 35 40

  • 20

20 40 60 80 100 120 140 12 14 16 18 20 22 24 26 28 30 32 34

Carnatic Music: A Computational Perspective

slide-25
SLIDE 25

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Extraction of Cent filterbank features5

Pre-emphasis Hamming window Discrete Fourier Transform Cent filter banks Discrete Cosine Transform Waveform

Log Cent filter bank energy values

Cepstral Coefficients

Carnatic Music: A Computational Perspective

slide-26
SLIDE 26

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Cent Filterbanks

  • Cent Filterbanks are similar to CQT (Constant Q transform) filters with a

difference – they are tonic normalised.

  • CQT is equivalent to Cent Filterbanks with a tonic of 2.
  • Padi Sarala will talk more about cent filterbanks and their applications to

segmentation motif recognition 6, stroke recognition7 and archival 8

6unpublished 7Akshay Ananthapadmanabhan, Juan Bello, Raghava Krishnan and Hema A Murthy, “Tonic independent stroke transcription of the mridangam,

AES, 2014

8Padi Sarala and Hema A Murthy, “Inter and intra segmentation of Carnatic music recordings for archival,” ISMIR 2013.

Carnatic Music: A Computational Perspective

slide-27
SLIDE 27

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Repeating the experiments of C V Raman9.

  • Strokes were played carefully
  • The number of strokes is 8
  • According to Raman there are 5 different modes – resonances
  • Therefore 5 basis vectors were estimated in NMF

9Akshay Ananthapadmanabhan, Ashwin Bellur and Hema A Murthy, “Modal analysis and transcription of strokes of the mridangam using

non-negative matrix factorisation,” ICASSP 2013, Vancouver,Canada Carnatic Music: A Computational Perspective

slide-28
SLIDE 28

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Modes of Mridangam and Activations for different strokes

Tones 20 40 60 80 100 120 50 100 FourthTone 20 40 60 80 100 120 100 200 ThirdTone 20 40 60 80 100 120 20 40 SecondTone 20 40 60 80 100 120 20 40 FirstTone 20 40 60 80 100 120 100 200 FifthTone

Figure: Modal Tones Figure: Strokes and their modes

Carnatic Music: A Computational Perspective

slide-29
SLIDE 29

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Transcription of strokes

  • Training of HMMs using activations
  • Testing on untranscribed data – use NMF activations to segment the strokes
  • Performance evaluation
  • An accuracy of about 87-89% was obtained on a controlled experiment.
  • Current work:
  • Tonic independence using CQT and cent filterbanks along with HMMs

(unpublished) – very good results – Sarala/Akshay will talk about this.

  • Dr. Umayalpuram Sivaraman is working with us on transcribing data for his tanis –

he has defined a number of new strokes – a task by itself.

  • Other upcoming artists are helping us transcribe various other artists.

Carnatic Music: A Computational Perspective

slide-30
SLIDE 30

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Pitch extraction using modified group delay functions10

  • A group delay based approach to pitch extraction.
  • Flattened magnitude spectrum is a sinusoid.
  • Estimate the frequency of the sinusoid using modified group delay functions.
  • Results are encouraging – Rajeev Rajan will give details.

10Rajeev Rajan and Hema A Murthy, “Group delay based melody monopitch extraction from music,” ICASSP 2013.

Carnatic Music: A Computational Perspective

slide-31
SLIDE 31

MIR Indian Music Preliminaries Tonic Gamak¯ as in Carnatic Music Cent filterbanks Identifying the strokes of the mridangam Pitch Extraction Conclusions

Conclusions

  • Different Aspects of Carnatic Music Analysis addressed
  • Tonic identification: Group delay based, NMF
  • Motif discovery: Rough Longest Common Subsequence
  • Segmentation of recordings of Carnatic Music
  • Transcription of mridangam
  • Future Work
  • Motif discovery using varnams, oneliners of songs
  • Does an Al¯

apana in CM have a rhythm?

  • Processing music uploaded into Music Brainz – convert many of the research ideas

to enhance MIR

  • Multipitch pitch tracking using phase.
  • Cognitive signal processing and music – signal processing and machine learning

used in tandem.

Carnatic Music: A Computational Perspective