Music genre classification using traditional and relational - - PowerPoint PPT Presentation

Music genre classification using traditional and relational approaches

Jorge Valverde-Rebaza, Aurea Soriano, Lilian Berton, Maria Cristina F. Oliveira and Alneu de Andrade Lopes

Laboratory of Computational Intelligence (LABIC) Laboratory of Visualization, Imaging and Computer Graphics (VICG) University of S˜ ao Paulo (USP) Brazil October 2014

Outline

1

Introduction

2

Method

3

Experimental Evaluation

4

Conclusion

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 2 / 20

Outline

1

Introduction

2

Method Music Features Graph construction Graph construction

3

Experimental Evaluation Datasets Experimental setup Results

4

Conclusion

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 3 / 20

Introduction

Many music collections, typically very large

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 4 / 20

Introduction

Many music collections, typically very large Manual music classification: a non-expert person can identify the genre of a music with 72% accuracy after hearing 3 seconds of it [Perrot and Gjerdigen, 1999].

Classifying a large collection demands time, effort and expertise Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 4 / 20

Introduction

Many music collections, typically very large Manual music classification: a non-expert person can identify the genre of a music with 72% accuracy after hearing 3 seconds of it [Perrot and Gjerdigen, 1999].

Classifying a large collection demands time, effort and expertise

Automatic music classification: Solutions achieve high accuracy (ranging from 63 to 84%) [Shao et al., 2004, Pampalk et al., 2005, Scaringella and Mlynek, 2005, Yaslan and Cataltepe, 2009, Poria et al., 2013]

require music files with the same size applied on controlled environments, i.e. without considering class imbalance employed traditional classifiers Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 4 / 20

Outline

1

Introduction

2

Method Music Features Graph construction Graph construction

3

Experimental Evaluation Datasets Experimental setup Results

4

Conclusion

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 5 / 20

Music feature extraction

Feature extracted from MIDI description

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20

Music feature extraction

Feature extracted from MIDI description (12)Histograms

Distance: DTW Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20

Music feature extraction

Feature extracted from MIDI description (12)Histograms

Distance: DTW

(8) Moments

(4) Melody and (4) Rhythm: the mean, standard deviation, entropy and uniformity Distance: Euclidean Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20

Music feature extraction

Feature extracted from MIDI description (12)Histograms

Distance: DTW

(8) Moments

(4) Melody and (4) Rhythm: the mean, standard deviation, entropy and uniformity Distance: Euclidean

(248)Structure

Identification of patterns from chord sequences. Generates vectors of different sizes. Distance: DTW Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20

Music feature extraction

Feature extracted from MIDI description (12)Histograms

Distance: DTW

(8) Moments

(4) Melody and (4) Rhythm: the mean, standard deviation, entropy and uniformity Distance: Euclidean

(248)Structure

Identification of patterns from chord sequences. Generates vectors of different sizes. Distance: DTW Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20

Graph construction

Graph construction techniques

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20

Graph construction

Graph construction techniques kNN

Connect each vertex only to its k nearest neighbors Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20

Graph construction

Graph construction techniques kNN

Connect each vertex only to its k nearest neighbors

mutual-kNN

Two vertices are connected only if the neighborhood pertinence condition is met by both Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20

Graph construction

Graph construction techniques kNN

Connect each vertex only to its k nearest neighbors

mutual-kNN

Two vertices are connected only if the neighborhood pertinence condition is met by both

regular-kNN

All the vertices have the same degree Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20

Graph construction

Graph construction techniques kNN

Connect each vertex only to its k nearest neighbors

mutual-kNN

Two vertices are connected only if the neighborhood pertinence condition is met by both

regular-kNN

All the vertices have the same degree Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20

Classifiers

Classifiers

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20

Classifiers

Classifiers Traditional

Map input data to a category Decision trees, na¨ ıve Bayes, neural networks, support vector machine, etc Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20

Classifiers

Classifiers Traditional

Map input data to a category Decision trees, na¨ ıve Bayes, neural networks, support vector machine, etc Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20

Classifiers

Classifiers Traditional

Map input data to a category Decision trees, na¨ ıve Bayes, neural networks, support vector machine, etc

Relational

Map relational input data to a category Weighted vote relational neighbor, network-only Bayes, probabilistic relational neighbor, network-only link based [Macskassy, 2007] Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20

Classifiers

Classifiers Traditional

Map input data to a category Decision trees, na¨ ıve Bayes, neural networks, support vector machine, etc

Relational

Map relational input data to a category Weighted vote relational neighbor, network-only Bayes, probabilistic relational neighbor, network-only link based [Macskassy, 2007] Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20

Outline

1

Introduction

2

Method Music Features Graph construction Graph construction

3

Experimental Evaluation Datasets Experimental setup Results

4

Conclusion

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 9 / 20

Datasets

Tabela: Music genre distribution for the music collection considered

Genre # Tracks Classical 31 Brazilian Backcountry 243 Pop/Rock 550 Jazz 95 Total 919

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 10 / 20

Experimental setup

Feature vectors Histogram Moments Structure

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 11 / 20

Experimental setup

Feature vectors Histogram Moments Structure Graph construction (1 ≤ k ≤ 15 ) kNN mutual-kNN regular-kN

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 11 / 20

Experimental setup

Feature vectors Histogram Moments Structure Graph construction (1 ≤ k ≤ 15 ) kNN mutual-kNN regular-kN Classifiers Traditional Relational

Decision tree (J48) Na¨ ıve Bayes (NB) Multilayer perceptron with backpropagation (MLP) Support vector machine (SMO) weighted vote relational neighbor (wvrn) network-only Bayes (no-Bayes) probabilistic relational neighbor (prn) network-only link-based mode-link (no-lb-mode) count-link (no-lb-count) binary-link (no-lb-binary) class-distribution-link (no-lb-distrib) Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 11 / 20

Traditional classification results

Tabela: Traditional classifiers performance measured by AUC

J48 NB MLP SMO Histogram 0.619 0.607 0.665 0.506 Moments 0.706 0.750 0.771 0.585 Structure 0.738 0.920 0.816 0.724 Average rank 2.667 2.000 1.333 4.000

1 2 3 4 MLP NB J48 SMO CD Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 12 / 20

Traditional classification results

Tabela: Traditional classifiers performance measured by AUC

J48 NB MLP SMO Histogram 0.619 0.607 0.665 0.506 Moments 0.706 0.750 0.771 0.585 Structure 0.738 0.920 0.816 0.724 Average rank 2.667 2.000 1.333 4.000

1 2 3 4 MLP NB J48 SMO CD

Figura: Post-hoc test results for traditional classifiers performance

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 12 / 20

Relational classification results: using kNN

Tabela: Relational classifiers performance evaluated by AUC in kNN networks

no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn Histogram 0.575 (k=11) 0.723 (k=9) 0.622 (k=2) 0.71 (k=8) 0.712 (k=9) 0.515 (k=1) 0.537 (k=1) Moments 0.547 (k=5) 0.635 (k=13) 0.575 (k=8) 0.644 (k=7) 0.644 (k=9) 0.563 (k=2) 0.571 (k=3) Structure 0.834 (k=7) 0.939 (k=14) 0.851 (k=4) 0.945 (k=14) 0.931 (k=15) 0.922 (k=15) 0.903 (k=9) Average rank 6.333 2.000 4.667 1.667 2.333 5.667 5.333 1 2 3 4 5 6 7 no-lb-distrib no-lb-count wvrn no-lb-binary prn no-Bayes no-lb-mode CD Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 13 / 20

Relational classification results: using kNN

Tabela: Relational classifiers performance evaluated by AUC in kNN networks

no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn Histogram 0.575 (k=11) 0.723 (k=9) 0.622 (k=2) 0.71 (k=8) 0.712 (k=9) 0.515 (k=1) 0.537 (k=1) Moments 0.547 (k=5) 0.635 (k=13) 0.575 (k=8) 0.644 (k=7) 0.644 (k=9) 0.563 (k=2) 0.571 (k=3) Structure 0.834 (k=7) 0.939 (k=14) 0.851 (k=4) 0.945 (k=14) 0.931 (k=15) 0.922 (k=15) 0.903 (k=9) Average rank 6.333 2.000 4.667 1.667 2.333 5.667 5.333 1 2 3 4 5 6 7 no-lb-distrib no-lb-count wvrn no-lb-binary prn no-Bayes no-lb-mode CD

Figura: Post-hoc test results for relational classifiers built on kNN networks

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 13 / 20

Relational classification results: using mutual-kNN

Tabela: Relational classifiers performance evaluated by AUC in mutual-kNN networks

no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn Histogram 0.621 (k=1) 0.712 (k=10) 0.626 (k=2) 0.735 (k=12) 0.727 (k=13) 0.555 (k=1) 0.571 (k=1) Moments 0.570 (k=1) 0.657 (k=14) 0.588 (k=2) 0.633 (k=15) 0.630 (k=14) 0.578 (k=1) 0.574 (k=1) Structure 0.864 (k=1) 0.955 (k=14) 0.818 (k=2) 0.963 (k=15) 0.964 (k=14) 0.913 (k=6) 0.902 (k=2) Average rank 6.000 2.333 5.000 1.667 2.000 5.333 5.667 1 2 3 4 5 6 7 no-lb-distrib wvrn no-lb-count no-lb-binary no-Bayes prn no-lb-mode CD Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 14 / 20

Relational classification results: using mutual-kNN

Tabela: Relational classifiers performance evaluated by AUC in mutual-kNN networks

no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn Histogram 0.621 (k=1) 0.712 (k=10) 0.626 (k=2) 0.735 (k=12) 0.727 (k=13) 0.555 (k=1) 0.571 (k=1) Moments 0.570 (k=1) 0.657 (k=14) 0.588 (k=2) 0.633 (k=15) 0.630 (k=14) 0.578 (k=1) 0.574 (k=1) Structure 0.864 (k=1) 0.955 (k=14) 0.818 (k=2) 0.963 (k=15) 0.964 (k=14) 0.913 (k=6) 0.902 (k=2) Average rank 6.000 2.333 5.000 1.667 2.000 5.333 5.667 1 2 3 4 5 6 7 no-lb-distrib wvrn no-lb-count no-lb-binary no-Bayes prn no-lb-mode CD

Figura: Post-hoc test results for relational classifiers built on mutual-kNN networks

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 14 / 20

Relational classification results: using regular-kNN

Tabela: Relational classifiers performance evaluated by AUC in regular-kNN networks

no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn Histogram 0.608 (k=1) 0.724 (k=8) 0.611 (k=2) 0.737 (k=12) 0.730 (k=12) 0.544 (k=1) 0.553 (k=1) Moments 0.569 (k=1) 0.652 (k=13) 0.571 (k=1) 0.620 (k=8) 0.625 (k=6) 0.560 (k=1) 0.565 (k=1) Structure 0.904 (k=1) 0.948 (k=11) 0.82 (k=1) 0.967 (k=15) 0.966 (k=15) 0.923 (k=3) 0.904 (k=2) Average rank 5.333 2.333 5.000 1.667 2.000 6.000 5.667 1 2 3 4 5 6 7 no-lb-distrib wvrn no-lb-count no-lb-binary no-lb-mode prn no-Bayes CD Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 15 / 20

Relational classification results: using regular-kNN

Tabela: Relational classifiers performance evaluated by AUC in regular-kNN networks

no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn Histogram 0.608 (k=1) 0.724 (k=8) 0.611 (k=2) 0.737 (k=12) 0.730 (k=12) 0.544 (k=1) 0.553 (k=1) Moments 0.569 (k=1) 0.652 (k=13) 0.571 (k=1) 0.620 (k=8) 0.625 (k=6) 0.560 (k=1) 0.565 (k=1) Structure 0.904 (k=1) 0.948 (k=11) 0.82 (k=1) 0.967 (k=15) 0.966 (k=15) 0.923 (k=3) 0.904 (k=2) Average rank 5.333 2.333 5.000 1.667 2.000 6.000 5.667 1 2 3 4 5 6 7 no-lb-distrib wvrn no-lb-count no-lb-binary no-lb-mode prn no-Bayes CD

Figura: Post-hoc test results for relational classifiers built on regular-kNN networks

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 15 / 20

Best performances

1 2 3 regular-kNN mutual-kNN kNN CD

Figura: Post-hoc test results for identifying the influence of network construction techniques in relational classifiers

1 2 3 4 no-lb-distrib wvrn MLP NB CD

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 16 / 20

Best performances

1 2 3 regular-kNN mutual-kNN kNN CD

Figura: Post-hoc test results for identifying the influence of network construction techniques in relational classifiers

1 2 3 4 no-lb-distrib wvrn MLP NB CD

Figura: Post-hoc test results for the comparison between the best traditional and relational classifiers

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 16 / 20

Outline

1

Introduction

2

Method Music Features Graph construction Graph construction

3

Experimental Evaluation Datasets Experimental setup Results

4

Conclusion

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 17 / 20

Conclusion

We introduce a novel feature vector (called Structural) which captures information encoded in the MIDI files

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20

Conclusion

We introduce a novel feature vector (called Structural) which captures information encoded in the MIDI files We evaluated traditional and relational classifiers on a music collection with an imbalanced distribution of four music genres

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20

Conclusion

We introduce a novel feature vector (called Structural) which captures information encoded in the MIDI files We evaluated traditional and relational classifiers on a music collection with an imbalanced distribution of four music genres The Structural features resulted in improved performance of both traditional and relational classifiers

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20

Conclusion

We introduce a novel feature vector (called Structural) which captures information encoded in the MIDI files We evaluated traditional and relational classifiers on a music collection with an imbalanced distribution of four music genres The Structural features resulted in improved performance of both traditional and relational classifiers The regular-kNN networks provided the relational model most suitable to improve the performance of relational classifiers

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20

Conclusion

We introduce a novel feature vector (called Structural) which captures information encoded in the MIDI files We evaluated traditional and relational classifiers on a music collection with an imbalanced distribution of four music genres The Structural features resulted in improved performance of both traditional and relational classifiers The regular-kNN networks provided the relational model most suitable to improve the performance of relational classifiers Relational classifiers perform better than traditional classifiers on music genre classification

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20

References

Macskassy, S. A. (2007). Improving learning in networked data by combining explicit and mined links. In AAAI, pages 590–595. Pampalk, E., Flexer, A., Widmer, G., et al. (2005). Improvements of audio-based music similarity and genre classificaton. In ISMIR, volume 5, pages 634–637. Perrot, D. and Gjerdigen, R. (1999). Scanning the dial: An exploration of factors in the identification of musical style. In Proceedings of the 1999 Society for Music Perception and Cognition, page 88. Poria, S., Gelbukh, A., Hussain, A., Bandyopadhyay, S., and Howard, N. (2013). Music genre classification: A semi-supervised approach. In Pattern Recognition, pages 254–263. Scaringella, N. and Mlynek, D. (2005). A mixture of support vector machines for audio classification. IEEE MIREX, London. Shao, X., Xu, C., and Kankanhalli, M. S. (2004). Unsupervised classification of music genre using hidden markov model. In Multimedia and Expo, 2004. ICME’04. 2004 IEEE International Conference on, volume 3, pages 2023–2026. Yaslan, Y. and Cataltepe, Z. (2009). Audio genre classification with semi-supervised feature ensemble learning. In Second International Workshop on Machine Learning and Music. Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 19 / 20

Thank you!

Jorge Valverde-Rebaza jvalverr@icmc.usp.br

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 20 / 20