Techniques de suivi de partition en temps réel Jonathan Aceituno Université Bordeaux 1 — Master 2 Informatique — Initiation à la recherche 1
Plan Introduction Premières techniques Déformation temporelle dynamique Modèles de Markov cachés Modélisation anticipative Conclusion 2
� � Suivi de partition � � � YOU ARE � � � � � HERE � � � � Partition Performance 3
� � Suivi de partition � � � HERE YOU � � � � � ARE � � � � Partition Performance 4
Problèmes Interprétation Alignement en temps réel Estimation du tempo ... 5
� � Alignement en temps réel � � � � � � � � � � � � 6
Historique mon article de base sur la modélisation anticipative Alignment of monophonic and polyphonic music to a score, Orio & Schwarz 2001 Score following using spectral analysis and hidden Markov models, Orio & Déchelle 2001 An online time warping algorithm for tracking musical performances, Dixon 2005 An online algorithm for real-time accompaniment, Dannenberg Score following at Ircam, Schwartz & al. 1984 2006 Training the synthetic performer, Vercoe & Puckette Artificially intelligent accompaniment using hidden Markov 1985 models to model musical structure, Jordanous & Smail 2008 New techniques for enhanced quality of computer accompaniment A coupled duration-focused architecture for realtime music to 1988 score alignment, Cont The synthetic performer, Vercoe Score following using the sung voice 2009 1984 1995 Real-time audio-to-score alignment using particle filter for Score following in practice, Puckette Score-performance matching using HMMs, Cano & al. coplayer music robots, Otsuka & al. 1990 1999 2011 7 1985 1990 1995 2000 2005 2010 2015 2020
Plan Introduction Premières techniques Déformation temporelle dynamique Modèles de Markov cachés Modélisation anticipative Conclusion 8
? Historique C 1 1 2 2 2 ! 1 2 3 3 E 1 2 3 3 Alignment of monophonic and polyphonic music to a score, Premières techniques Orio & Schwarz 2001 Score following using spectral analysis and hidden Markov models, Orio & Déchelle 2001 An online time warping algorithm for tracking musical performances, Dixon 2005 An online algorithm for real-time accompaniment, Dannenberg Score following at Ircam, Schwartz & al. 1984 2006 Training the synthetic performer, Vercoe & Puckette Artificially intelligent accompaniment using hidden Markov 1985 models to model musical structure, Jordanous & Smail 2008 New techniques for enhanced quality of computer accompaniment A coupled duration-focused architecture for realtime music to 1988 score alignment, Cont The synthetic performer, Vercoe Score following using the sung voice 2009 1984 1995 Real-time audio-to-score alignment using particle filter for Score following in practice, Puckette Score-performance matching using HMMs, Cano & al. coplayer music robots, Otsuka & al. 1990 1999 2011 9 1985 1990 1995 2000 2005 2010 2015 2020
Premières techniques ? C 1 1 � � 2 2 2 ! 1 2 3 3 E 1 2 3 3 Traitement séquentiel des événements (Vercoe 84, Puckette 92, Puckette 95...) � � � � � � � � � � � � � � � � [ , , , , , ...] � � � � � � � � = � � Programmation dynamique (Dannenberg 84, Dannenberg & Mukaino 88...) performance F A G # C F partition 1 1 1 1 A 2 2 2 1 G # 1 2 3 3 E 2 1 3 3 10
Historique Alignment of monophonic and polyphonic music to a score, Déformation temporelle Orio & Schwarz 2001 dynamique Score following using spectral analysis and hidden Markov models, Orio & Déchelle 2001 An online time warping algorithm for tracking musical performances, Dixon 2005 An online algorithm for real-time accompaniment, Dannenberg Score following at Ircam, Schwartz & al. 1984 2006 Training the synthetic performer, Vercoe & Puckette Artificially intelligent accompaniment using hidden Markov 1985 models to model musical structure, Jordanous & Smail 2008 New techniques for enhanced quality of computer accompaniment A coupled duration-focused architecture for realtime music to 1988 score alignment, Cont The synthetic performer, Vercoe Score following using the sung voice 2009 1984 1995 Real-time audio-to-score alignment using particle filter for Score following in practice, Puckette Score-performance matching using HMMs, Cano & al. coplayer music robots, Otsuka & al. 1990 1999 2011 11 1985 1990 1995 2000 2005 2010 2015 2020
Déformation temporelle dynamique alignement optimal (Orio & al. 01, Dixon 05...) partition Programmation dynamique pour l’audio Idée : performance ≈ partition position actuelle déformée dans le temps Découpage du signal en trames Séquences à aligner : spectres ou vecteurs de descripteurs performance Choix de la distance 12
Historique q 1 q 2 y 1 y 2 Alignment of monophonic and polyphonic music to a score, Modèles de Markov Orio & Schwarz 2001 cachés Score following using spectral analysis and hidden Markov models, Orio & Déchelle 2001 An online time warping algorithm for tracking musical performances, Dixon 2005 An online algorithm for real-time accompaniment, Dannenberg Score following at Ircam, Schwartz & al. 1984 2006 Training the synthetic performer, Vercoe & Puckette Artificially intelligent accompaniment using hidden Markov 1985 models to model musical structure, Jordanous & Smail 2008 New techniques for enhanced quality of computer accompaniment A coupled duration-focused architecture for realtime music to 1988 score alignment, Cont The synthetic performer, Vercoe Score following using the sung voice 2009 1984 1995 Real-time audio-to-score alignment using particle filter for Score following in practice, Puckette Score-performance matching using HMMs, Cano & al. coplayer music robots, Otsuka & al. 1990 1999 2011 13 1985 1990 1995 2000 2005 2010 2015 2020
Modèles de Markov cachés q 1 q 2 y 1 y 2 (Cano & al. 99, Orio & al. 01, Schwartz & al. 06, Jordanous & al. 08...) Approche générative : modèle = partition Sources d’incertitude Erreurs du musicien Fiabilité des descripteurs Modèles probabilistes intéressants 14
Modèles de Markov cachés q 1 q 2 y 1 y 2 0.5 0.2 0.5 Chaîne de Markov : matrice de transition A=(a ij ) 0.8 Probabilités d’observation b j pour chaque état matrice de sobre ivre À chaque instant, changement d’état transition 0.5 0.5 sobre de i à j selon (a ij ), l’état courant j 0.8 0.2 ivre génère une observation selon b j probabilités sobre ivre d’observation 0.3 0.6 ronfle ne ronfle 0.7 0.4 pas 15
Algorithme de Viterbi q 1 q 2 y 1 y 2 Problème : retrouver la suite d’états qui a généré les observations Idée : calcul des probabilités max δ t (i) entre les chemins d’états finissant par i à t Astuce : formulation récursive de δ t (i) ⇒ programmation dynamique 16
Modèles de Markov cachés q 1 q 2 y 1 y 2 � � � � � � � MMC gauche-droite = partition Algorithme de Viterbi ⇒ états ⇒ position 17
Historique q 1 q 2 Alignment of monophonic and polyphonic music to a score, Modélisation Orio & Schwarz 2001 anticipative Score following using spectral analysis and hidden Markov models, Orio & Déchelle 2001 An online time warping algorithm for tracking musical performances, Dixon 2005 An online algorithm for real-time accompaniment, Dannenberg Score following at Ircam, Schwartz & al. 1984 2006 Training the synthetic performer, Vercoe & Puckette Artificially intelligent accompaniment using hidden Markov 1985 models to model musical structure, Jordanous & Smail 2008 New techniques for enhanced quality of computer accompaniment A coupled duration-focused architecture for realtime music to 1988 score alignment, Cont The synthetic performer, Vercoe Score following using the sung voice 2009 1984 1995 Real-time audio-to-score alignment using particle filter for Score following in practice, Puckette Score-performance matching using HMMs, Cano & al. coplayer music robots, Otsuka & al. 1990 1999 2011 18 1985 1990 1995 2000 2005 2010 2015 2020
Modélisation anticipative q 1 q 2 (Cont 09) Problème des MMC : distribution du temps de séjour d j implicite Modèles semi-Markov cachés Problèmes des MSMC : algorithmes trop coûteux Modèles hybrides Markov/semi- Markov 19
Modélisation anticipative q 1 q 2 Pourquoi ce nom, alors ? anticipation = action(passé, prédictions) (Cont 08) Modéliser l’anticipation Modélisation anticipative 20
Modélisation anticipative q 1 q 2 Mécanisme anticipatif : couplage position actuelle MHMSMC Tempo temps de séjour traitement oscillateur performance interne 21
Plan Introduction Premières techniques Déformation temporelle dynamique Modèles de Markov cachés Modélisation anticipative Conclusion 22
Conclusion Meilleurs résultats en général : modélisation anticipative Problème de l’évaluation Évolution du domaine ? 23
Courage ! C’est bientôt l’heure du goûter ! Merci de votre attention 24
Recommend
More recommend
