Gestural input effects on the spectral envelope of violin sounds - PowerPoint PPT Presentation

Gestural input effects 
 on the spectral envelope 
 of violin sounds Quim Llimona MUMT 605 Final Project December 18th, 2015

Background: MUSMAP

Background: MUSMAP

Background: MUSMAP

What does a violin note look like? 0.1 0.02 Waveform Waveform 0 0 − 0.02 − 0.1 − 0.04 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 20 20 15 Spectrogram 15 Spectrogram 10 10 5 5 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1 0.4 Bow velocity Bow velocity 0.5 0.2 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1 0.5 Bow force Bow force 0.45 0.5 0.4 0.35 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

What does a violin note look like? 0.1 0.02 Waveform Waveform 0 0 − 0.02 − 0.1 − 0.04 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 20 20 15 Spectrogram 15 Spectrogram 10 10 5 5 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1 0.4 Bow velocity Bow velocity 0.5 0.2 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1 0.5 Bow force Bow force 0.45 0.5 0.4 0.35 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Spectral envelope extraction: LPC LPC (p=8) LPC (p=16) 40 40 20 20 Power (dB) Power (dB) 0 0 − 20 − 20 − 40 − 40 − 60 − 60 − 80 − 80 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Frequency (Hz) Frequency (Hz) 4 4 x 10 x 10 LPC (p=32) LPC (p=64) 40 40 20 20 Power (dB) Power (dB) 0 0 − 20 − 20 − 40 − 40 − 60 − 60 − 80 − 80 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Frequency (Hz) Frequency (Hz) 4 4 x 10 x 10

Spectral envelope extraction: PEAKPICK 40 20 0 Magnitude (dB) − 20 − 40 − 60 − 80 0 0.5 1 1.5 2 Frequency (Hz) 4 x 10

Spectral envelope extraction: MAXPOOL 40 40 MAXPOOL (n=218) MAXPOOL (n=72) 20 20 Power (dB) Power (dB) 0 0 − 20 − 20 − 40 − 40 − 60 − 60 − 80 − 80 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Frequency (Hz) Frequency (Hz) 4 4 x 10 x 10 40 40 MAXPOOL (n=55) MAXPOOL (n=22) 20 20 Power (dB) Power (dB) 0 0 − 20 − 20 − 40 − 40 − 60 − 60 − 80 − 80 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Frequency (Hz) Frequency (Hz) 4 4 x 10 x 10

Spectral whitening 
 Admittance deconvolution Deconvolved Raw 80 40 60 20 40 0 Power (dB) Power (dB) 20 − 20 0 − 40 − 20 − 60 − 40 − 80 0 0.5 1 1.5 2 0 0.5 1 1.5 2 Frequency (Hz) Frequency (Hz) 4 4 x 10 x 10

Spectral whitening Data-driven normalization 20 20 Magnitude (dB) Magnitude (dB) 0 0 − 20 − 20 − 40 − 40 Original Original Whitened Whitened − 60 − 60 3 4 3 4 10 10 10 10 Frequency (Hz) Frequency (Hz) 20 20 Magnitude (dB) Magnitude (dB) 0 0 − 20 − 20 − 40 − 40 Original Original Whitened Whitened − 60 − 60 3 4 3 4 10 10 10 10 Frequency (Hz) Frequency (Hz)

Mean bow force Mean bow velocity Mean bow bridge distance 15 20 15 15 10 10 10 5 5 5 0 0 0 0.2 0.4 0.6 0.8 1 1.2 0.4 0.6 0.8 1 1.2 1.4 20 30 40 50 9267 9267 9267 Frequency (Hz) Frequency (Hz) Frequency (Hz) 3552 3552 3552 1361 1361 1361 522 522 522 200 200 200 0.0 0.2 9 Original Original Original 9267 9267 9267 Frequency (Hz) Frequency (Hz) Frequency (Hz) 3552 3552 3552 1361 1361 1361 522 522 522 200 200 200 0.0 0.2 9 Deconvolved Deconvolved Deconvolved

Learn convolutive templates through Non-Negative Matrix Factorization ( NMF ) on the log-spectrum (where chained filters are additive) 1 0.8 0.6 0.4 0.2 0 3 4 10 10 Frequency (Hz)

Distribution of NMF activations

Distribution of NMF activations

Distribution of NMF activations

Ratio and sum of NMF activations

Thanks! Quim Llimona MUMT 605 Final Project December 18th, 2015