Unsupervised Piano Music Transcription Taylor Berg-Kirkpatrick - - PowerPoint PPT Presentation

▶
unsupervised piano music transcription
SMART_READER_LITE
LIVE PREVIEW

Unsupervised Piano Music Transcription Taylor Berg-Kirkpatrick - - PowerPoint PPT Presentation

Jun 09, 2023 317 likes •891 views

Unsupervised Piano Music Transcription Taylor Berg-Kirkpatrick Jacob Andreas and Dan Klein CMU UC Berkeley Piano Music Transcription note time Supervised Transcription Supervised Transcription w Model

slide-1
SLIDE 1

Unsupervised Piano Music Transcription

Taylor Berg-Kirkpatrick Jacob Andreas and Dan Klein CMU UC Berkeley

slide-2
SLIDE 2

Piano Music Transcription

time note

slide-3
SLIDE 3

Supervised Transcription

slide-4
SLIDE 4

Supervised Transcription

Model

w

slide-5
SLIDE 5

Supervised Transcription

slide-6
SLIDE 6

Learning

w

Learning to Transcribe

slide-7
SLIDE 7

Prediction

Model

w

?

slide-8
SLIDE 8

Prediction

Model

w

slide-9
SLIDE 9

Piano Sounds

slide-10
SLIDE 10

Piano Sounds

slide-11
SLIDE 11

Piano Sounds

slide-12
SLIDE 12

Piano Sounds

slide-13
SLIDE 13

Piano Sounds

slide-14
SLIDE 14

Piano Sounds

slide-15
SLIDE 15

Piano Sounds

time freq

slide-16
SLIDE 16

Spectral Shape

time freq

slide-17
SLIDE 17

Spectral Shape

time freq

slide-18
SLIDE 18

Spectral Shape

time freq

slide-19
SLIDE 19

Spectral Shape

time freq

slide-20
SLIDE 20

Temporal Shape

time freq

slide-21
SLIDE 21

Temporal Shape

time freq

slide-22
SLIDE 22

Temporal Shape

time freq

slide-23
SLIDE 23

Temporal Shape

time freq

slide-24
SLIDE 24

Temporal Shape

time freq

slide-25
SLIDE 25

Temporal Shape

time freq

slide-26
SLIDE 26

Temporal Shape

time freq

slide-27
SLIDE 27

Temporal Shape

time freq

slide-28
SLIDE 28

Temporal Shape

time freq

slide-29
SLIDE 29

Temporal Shape

time freq

slide-30
SLIDE 30

Temporal Shape

time freq

slide-31
SLIDE 31

Temporal Shape

time freq

slide-32
SLIDE 32

Polyphony

slide-33
SLIDE 33

Polyphony

. . .

slide-34
SLIDE 34

Unsupervised Transcription

slide-35
SLIDE 35

Unsupervised Transcription

Symbolic Music

Audio signal

slide-36
SLIDE 36

Learning Unsupervised Transcription

Piano Parameters Symbolic Music

Generative Model

Audio signal

? ?

slide-37
SLIDE 37

Learning Unsupervised Transcription

Piano Parameters Symbolic Music

Generative Model

Audio signal

? ?

slide-38
SLIDE 38

Generative Model

time note

Mn

n

Note events

time velocity

slide-39
SLIDE 39

Generative Model

Note events

time

Activation

time

Spectrogram

time freq

Component spectrogram

time freq freq time

duration velocity

REST PLAY

X

Parameters Latent variables

Sn An Mn

µn αn σn

slide-40
SLIDE 40

E1 E2 E3

V2

V3

Event type

V1

Velocity

PLAY REST PLAY

duration velocity

REST PLAY

Note Event Model

D1

Duration

D2 D3

Mn

µn

slide-41
SLIDE 41

Activation Model

V2

V3

V1 D1 D2 D3

slide-42
SLIDE 42

An

Activation

Activation Model

V2 V3 V1 D1 D2 D3

copy temporal shape αn

Temporal shape

slide-43
SLIDE 43

An

Activation

Activation Model

V2 V3 V1 D1 D2 D3

αn truncate to duration

Temporal shape

slide-44
SLIDE 44

An

Activation

Activation Model

V2 V3 V1 D1 D2 D3

αn scale to velocity

Temporal shape

slide-45
SLIDE 45

An

Activation

Activation Model

V2 V3 V1 D1 D2 D3

αn add Gaussian noise

Temporal shape

slide-46
SLIDE 46

Component Spectrogram Model

An

Activation

σn

Spectral shape

Sn

Component spectrogram

Poisson noise

slide-47
SLIDE 47

Total Spectrogram Model

. . .

σ1 σN S1 SN

Total spectrogram

+

X

A1 AN

slide-48
SLIDE 48

Learning and Inference

Note events

time

Activation

time

Spectrogram

time freq

Component spectrogram

time freq freq time

duration velocity

REST PLAY

X

Parameters Latent variables

Sn An Mn

µn αn σn

slide-49
SLIDE 49

Learning and Inference

Semi-Markov dynamic program Closed form update Exponentiated gradient ascent Exponentiated gradient ascent

α|A, M

Temporal shapes update:

A|M, X, α, σ

Activations update:

σ|A, X

Spectral shapes update: Note events update:

M|A, α, µ

slide-50
SLIDE 50

Evaluation

Onset F1

time note

slide-51
SLIDE 51

Results

50 60 70 80 82.1 70.4

69.0

68.6 58.3

Onset F1

MAPS Corpus

O’Hanlon 2014 Benetos 2014 Vincent 2013

Unsupervised*

[Berg-Kirkpatrick et al. 2014] [Valentin et al. 2010]

Supervised

slide-52
SLIDE 52

Transcription

Predicted Reference

slide-53
SLIDE 53

Resynthesized Examples

Grieg input Grieg resynth piano Grieg resynth guitar

slide-54
SLIDE 54

Demo

Demo!

slide-55
SLIDE 55

Resynthesized Examples

Chopin input Chopin resynth piano Chopin resynth guitar

slide-56
SLIDE 56

Resynthesized Examples

Beethoven input Beethoven resynth piano Beethoven resynth guitar