detection of laughter in interaction in multichannel
play

Detection of Laughter-in-Interaction in Multichannel Close-Talk - PowerPoint PPT Presentation

Introduction Data Model Experiments Analysis Summary Detection of Laughter-in-Interaction in Multichannel Close-Talk Microphone Recordings of Meetings Kornel Laskowski and Tanja Schultz Cognitive Systems Lab, Universit at Karlsruhe


  1. Introduction Data Model Experiments Analysis Summary Detecting All Laughter from All Audio 1 2 3 inactive (don’t decode this channel) 4 Past work has focused on: a subset of laughter (improving recall) isolated laughter loud, clear, unambiguous laughter and/or a subset of audio (improving precision) segmented intervals only active channels K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  2. Introduction Data Model Experiments Analysis Summary Brief Comparison with Related Work L / S class. L / ¬L segm. this Aspect [1] [2] [3] [4] [5] work close-talk microphones � � � � � farfield microphones � single channel at-a-time � � � � multi-channel at-a-time � � participant attribution � � � � � only group laughter � only isolated laughter � � � only clear laughter � rely on pre-segmentation � � ? rely on channel exclusion ? � [1] (Truong & van Leeuwen, 2005); [2] (Truong & van Leeuwen, 2007a); [3] (Truong & van Leeuwen, 2007b); [4] (Knox & Mirghafori, 2007); [5] (Kennedy & Ellis, 2004). K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  3. Introduction Data Model Experiments Analysis Summary Outline of this Talk 1. Introduction (about to be over) 2. Data 3. Multiparticipant 3-state Vocal Activity Detector 4. Experiments 5. Analysis 6. Conclusions (& Unqualified Recommendations) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  4. Introduction Data Model Experiments Analysis Summary ICSI Meeting Corpus the complete corpus (Janin et al, 2003) 75 naturally occurring meetings longitudinal CTM recordings of several work groups 3-9 instrumented participants per meeting we use a subset of 67 meetings types: Bed (15), Bmr (29), Bro (23) 23 unique participants 3 participants attend both Bmr and Bro 1 participant attends both Bmr and Bed in particular, as elsewhere, TrainSet : 26 Bmr meetings TestSet : 3 Bmr meetings K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  5. Introduction Data Model Experiments Analysis Summary ICSI Meeting Corpus the complete corpus (Janin et al, 2003) 75 naturally occurring meetings longitudinal CTM recordings of several work groups 3-9 instrumented participants per meeting we use a subset of 67 meetings types: Bed (15), Bmr (29), Bro (23) 23 unique participants 3 participants attend both Bmr and Bro 1 participant attends both Bmr and Bed in particular, as elsewhere, TrainSet : 26 Bmr meetings TestSet : 3 Bmr meetings K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  6. Introduction Data Model Experiments Analysis Summary ICSI Meeting Corpus the complete corpus (Janin et al, 2003) 75 naturally occurring meetings longitudinal CTM recordings of several work groups 3-9 instrumented participants per meeting we use a subset of 67 meetings types: Bed (15), Bmr (29), Bro (23) 23 unique participants 3 participants attend both Bmr and Bro 1 participant attends both Bmr and Bed in particular, as elsewhere, TrainSet : 26 Bmr meetings TestSet : 3 Bmr meetings K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  7. Introduction Data Model Experiments Analysis Summary Reference Segmentation speech, S forced alignment of words and word fragments available in the ICSI MRDA Corpus (Shriberg et al, 2004) bridge inter-lexeme gaps shorter than 300 ms as in NIST Rich Transcription Meeting Recognition evaluations laughter, L produced semi-automatically (Laskowski & Burger, 2007d) ≥ 99% of laughter markup, as originally transcribed bouts include terminal “recovery” in-/ehxalation, if present augmented with voicing classification, L ≡ L V ∪ L U “laughed speech” (Nwokah et al, 1999) , S ∩ L here, mapped to laughter L each participant can be producing L , S , or neither K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  8. Introduction Data Model Experiments Analysis Summary Reference Segmentation speech, S forced alignment of words and word fragments available in the ICSI MRDA Corpus (Shriberg et al, 2004) bridge inter-lexeme gaps shorter than 300 ms as in NIST Rich Transcription Meeting Recognition evaluations laughter, L produced semi-automatically (Laskowski & Burger, 2007d) ≥ 99% of laughter markup, as originally transcribed bouts include terminal “recovery” in-/ehxalation, if present augmented with voicing classification, L ≡ L V ∪ L U “laughed speech” (Nwokah et al, 1999) , S ∩ L here, mapped to laughter L each participant can be producing L , S , or neither K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  9. Introduction Data Model Experiments Analysis Summary Reference Segmentation speech, S forced alignment of words and word fragments available in the ICSI MRDA Corpus (Shriberg et al, 2004) bridge inter-lexeme gaps shorter than 300 ms as in NIST Rich Transcription Meeting Recognition evaluations laughter, L produced semi-automatically (Laskowski & Burger, 2007d) ≥ 99% of laughter markup, as originally transcribed bouts include terminal “recovery” in-/ehxalation, if present augmented with voicing classification, L ≡ L V ∪ L U “laughed speech” (Nwokah et al, 1999) , S ∩ L here, mapped to laughter L each participant can be producing L , S , or neither K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  10. Introduction Data Model Experiments Analysis Summary Reference Segmentation speech, S forced alignment of words and word fragments available in the ICSI MRDA Corpus (Shriberg et al, 2004) bridge inter-lexeme gaps shorter than 300 ms as in NIST Rich Transcription Meeting Recognition evaluations laughter, L produced semi-automatically (Laskowski & Burger, 2007d) ≥ 99% of laughter markup, as originally transcribed bouts include terminal “recovery” in-/ehxalation, if present augmented with voicing classification, L ≡ L V ∪ L U “laughed speech” (Nwokah et al, 1999) , S ∩ L here, mapped to laughter L each participant can be producing L , S , or neither K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  11. Introduction Data Model Experiments Analysis Summary Multiparticipant 3-state Vocal Activity Detector hidden Markov model pruned Viterbi (beam) decoding topology single participant state subspace multiparticipant state space, pruning multiparticipant transition probability model standard MFCC features, plus crosstalk suppression features multiparticipant emission probability model K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  12. Introduction Data Model Experiments Analysis Summary Single Participant (SP) State Subspace L (+3) frame step ∆ T = 0 . 1 s L (+3) L (+2) explicit minimum duration L (+4) constraints N ( − 3) N ( − 2) L (+1) T S min , T L min , T N � � ≡ T min N ( − 1) min N S min , N L min , N N � � = ∆ T · N ( − 1) N (0) S (+2) min S (+1) number of states in 1-participant S (+4) subspace S (+2) each participant can be N S min + N L min + N N N = min S (+3) speaking, S laughing, L in example shown, N = 9 silent, N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  13. Introduction Data Model Experiments Analysis Summary Single Participant (SP) State Subspace L (+3) frame step ∆ T = 0 . 1 s L (+3) L (+2) explicit minimum duration L (+4) constraints N ( − 3) N ( − 2) L (+1) T S min , T L min , T N � � ≡ T min N ( − 1) min N S min , N L min , N N � � = ∆ T · N ( − 1) N (0) S (+2) min S (+1) number of states in 1-participant S (+4) subspace S (+2) each participant can be N S min + N L min + N N N = min S (+3) speaking, S laughing, L in example shown, N = 9 silent, N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  14. Introduction Data Model Experiments Analysis Summary Single Participant (SP) State Subspace L (+3) frame step ∆ T = 0 . 1 s L (+3) L (+2) explicit minimum duration L (+4) constraints N ( − 3) N ( − 2) L (+1) T S min , T L min , T N � � ≡ T min N ( − 1) min N S min , N L min , N N � � = ∆ T · N ( − 1) N (0) S (+2) min S (+1) number of states in 1-participant S (+4) subspace S (+2) each participant can be N S min + N L min + N N N = min S (+3) speaking, S laughing, L in example shown, N = 9 silent, N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  15. Introduction Data Model Experiments Analysis Summary Single Participant (SP) State Subspace L (+3) frame step ∆ T = 0 . 1 s L (+3) L (+2) explicit minimum duration L (+4) constraints N ( − 3) N ( − 2) L (+1) T S min , T L min , T N � � ≡ T min N ( − 1) min N S min , N L min , N N � � = ∆ T · N ( − 1) N (0) S (+2) min S (+1) number of states in 1-participant S (+4) subspace S (+2) each participant can be N S min + N L min + N N N = min S (+3) speaking, S laughing, L in example shown, N = 9 silent, N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  16. Introduction Data Model Experiments Analysis Summary Single Participant (SP) State Subspace L (+3) frame step ∆ T = 0 . 1 s L (+3) L (+2) explicit minimum duration L (+4) constraints N ( − 3) N ( − 2) L (+1) T S min , T L min , T N � � ≡ T min N ( − 1) min N S min , N L min , N N � � = ∆ T · N ( − 1) N (0) S (+2) min S (+1) number of states in 1-participant S (+4) subspace S (+2) each participant can be N S min + N L min + N N N = min S (+3) speaking, S laughing, L in example shown, N = 9 silent, N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  17. Introduction Data Model Experiments Analysis Summary Multiparticipant (MP) State Space for a conversation of K participants, form the Cartesian product of K factors: L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+2) L (+2) L (+2) L (+2) × × × · · · × L (+4) L (+4) L (+4) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) S (+1) S (+1) S (+1) S (+1) S (+4) S (+4) S (+4) S (+4) S (+2) S (+2) S (+2) S (+2) each MP state: K -vector of N SP states S (+3) S (+3) S (+3) S (+3) total number of MP states in topology: N K impose maximum simultaneous vocalization constraints K S max , K L max , K ¬N � � = K max max ie. K L max : max. # participants laughing at the same time K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  18. Introduction Data Model Experiments Analysis Summary Multiparticipant (MP) State Space for a conversation of K participants, form the Cartesian product of K factors: L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+2) L (+2) L (+2) L (+2) × × × · · · × L (+4) L (+4) L (+4) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) S (+1) S (+1) S (+1) S (+1) S (+4) S (+4) S (+4) S (+4) S (+2) S (+2) S (+2) S (+2) each MP state: K -vector of N SP states S (+3) S (+3) S (+3) S (+3) total number of MP states in topology: N K impose maximum simultaneous vocalization constraints K S max , K L max , K ¬N � � = K max max ie. K L max : max. # participants laughing at the same time K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  19. Introduction Data Model Experiments Analysis Summary Multiparticipant (MP) State Space for a conversation of K participants, form the Cartesian product of K factors: L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+2) L (+2) L (+2) L (+2) × × × · · · × L (+4) L (+4) L (+4) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) S (+1) S (+1) S (+1) S (+1) S (+4) S (+4) S (+4) S (+4) S (+2) S (+2) S (+2) S (+2) each MP state: K -vector of N SP states S (+3) S (+3) S (+3) S (+3) total number of MP states in topology: N K impose maximum simultaneous vocalization constraints K S max , K L max , K ¬N � � = K max max ie. K L max : max. # participants laughing at the same time K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  20. Introduction Data Model Experiments Analysis Summary Multiparticipant (MP) State Space for a conversation of K participants, form the Cartesian product of K factors: L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+3) L (+2) L (+2) L (+2) L (+2) × × × · · · × L (+4) L (+4) L (+4) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) S (+1) S (+1) S (+1) S (+1) S (+4) S (+4) S (+4) S (+4) S (+2) S (+2) S (+2) S (+2) each MP state: K -vector of N SP states S (+3) S (+3) S (+3) S (+3) total number of MP states in topology: N K impose maximum simultaneous vocalization constraints K S max , K L max , K ¬N � � = K max max ie. K L max : max. # participants laughing at the same time K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  21. Introduction Data Model Experiments Analysis Summary Transition Probability Model Example, K = 5: � S (2) , N (0) , N (0) , N (0) � at time t , q t = S i = N ( − 2) , N (0) , S (1) , L (1) � � at time t + 1, q t +1 = S j = what is a ij = P ( q t +1 = S j | q t = S i ) ? 1 a ij = 0 if the SP transition from S i to S j for any participant is not licensed by the SP topology 2 otherwise, ML estimate using ngram counts from best flat-start Viterbi path over training corpus 3 NOTE: each participant’s index k in S is arbitrary for all K -symbol permutations/rotations R want P ( S j | S i ) ≡ P ( R · S j | R · S i ) during model training & querying, rotate each q t into a fixed ordering of the N single-participant states K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  22. Introduction Data Model Experiments Analysis Summary Transition Probability Model Example, K = 5: � S (2) , N (0) , N (0) , N (0) � at time t , q t = S i = N ( − 2) , N (0) , S (1) , L (1) � � at time t + 1, q t +1 = S j = what is a ij = P ( q t +1 = S j | q t = S i ) ? 1 a ij = 0 if the SP transition from S i to S j for any participant is not licensed by the SP topology 2 otherwise, ML estimate using ngram counts from best flat-start Viterbi path over training corpus 3 NOTE: each participant’s index k in S is arbitrary for all K -symbol permutations/rotations R want P ( S j | S i ) ≡ P ( R · S j | R · S i ) during model training & querying, rotate each q t into a fixed ordering of the N single-participant states K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  23. Introduction Data Model Experiments Analysis Summary Transition Probability Model Example, K = 5: � S (2) , N (0) , N (0) , N (0) � at time t , q t = S i = N ( − 2) , N (0) , S (1) , L (1) � � at time t + 1, q t +1 = S j = what is a ij = P ( q t +1 = S j | q t = S i ) ? 1 a ij = 0 if the SP transition from S i to S j for any participant is not licensed by the SP topology 2 otherwise, ML estimate using ngram counts from best flat-start Viterbi path over training corpus 3 NOTE: each participant’s index k in S is arbitrary for all K -symbol permutations/rotations R want P ( S j | S i ) ≡ P ( R · S j | R · S i ) during model training & querying, rotate each q t into a fixed ordering of the N single-participant states K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  24. Introduction Data Model Experiments Analysis Summary Transition Probability Model Example, K = 5: � S (2) , N (0) , N (0) , N (0) � at time t , q t = S i = N ( − 2) , N (0) , S (1) , L (1) � � at time t + 1, q t +1 = S j = what is a ij = P ( q t +1 = S j | q t = S i ) ? 1 a ij = 0 if the SP transition from S i to S j for any participant is not licensed by the SP topology 2 otherwise, ML estimate using ngram counts from best flat-start Viterbi path over training corpus 3 NOTE: each participant’s index k in S is arbitrary for all K -symbol permutations/rotations R want P ( S j | S i ) ≡ P ( R · S j | R · S i ) during model training & querying, rotate each q t into a fixed ordering of the N single-participant states K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  25. Introduction Data Model Experiments Analysis Summary Observables each of K participants is wearing a close-talk mic (CTM) extract 41 features from every CTM channel log energy + MFCCs ∆s and ∆∆s min and max normalized log energy differences (NLEDs) (Boakye & Stolcke, 2006) 41 · K features per frame may vary from meeting to meeting (as K does) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  26. Introduction Data Model Experiments Analysis Summary Observables each of K participants is wearing a close-talk mic (CTM) extract 41 features from every CTM channel log energy + MFCCs ∆s and ∆∆s min and max normalized log energy differences (NLEDs) (Boakye & Stolcke, 2006) 41 · K features per frame may vary from meeting to meeting (as K does) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  27. Introduction Data Model Experiments Analysis Summary Observables each of K participants is wearing a close-talk mic (CTM) extract 41 features from every CTM channel log energy + MFCCs ∆s and ∆∆s min and max normalized log energy differences (NLEDs) (Boakye & Stolcke, 2006) 41 · K features per frame may vary from meeting to meeting (as K does) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  28. Introduction Data Model Experiments Analysis Summary Observables each of K participants is wearing a close-talk mic (CTM) extract 41 features from every CTM channel log energy + MFCCs ∆s and ∆∆s min and max normalized log energy differences (NLEDs) (Boakye & Stolcke, 2006) 41 · K features per frame may vary from meeting to meeting (as K does) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  29. Introduction Data Model Experiments Analysis Summary Emission Probability Model variable feature length vector X = [ X 1 , X 2 , · · · , X K ] train a single-channel GMM (64 components) for S and L for N all and N nearfield then approximate the joint MP emission with K � P ( X | S i ) = P ( X [ k ] | S i [ k ] ) (1) k =1 K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  30. Introduction Data Model Experiments Analysis Summary Emission Probability Model variable feature length vector X = [ X 1 , X 2 , · · · , X K ] train a single-channel GMM (64 components) for S and L for N all and N nearfield then approximate the joint MP emission with K � P ( X | S i ) = P ( X [ k ] | S i [ k ] ) (1) k =1 K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  31. Introduction Data Model Experiments Analysis Summary Emission Probability Model variable feature length vector X = [ X 1 , X 2 , · · · , X K ] train a single-channel GMM (64 components) for S and L for N all and N nearfield then approximate the joint MP emission with K � P ( X | S i ) = P ( X [ k ] | S i [ k ] ) (1) k =1 K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  32. Introduction Data Model Experiments Analysis Summary Described Experiments 1 independent versus joint participant decoding 2 sensitivity to minimum duration constraints 3 sensitivity to maximum overlap constraints 4 generalization to other (non- Bmr ) meetings Evaluation: recall (R), precision (P), and unweighted F goal here: L versus ¬L = S ∪ N sanity: S versus ¬S = L ∪ N sanity: V = S ∪ L versus ¬V = N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  33. Introduction Data Model Experiments Analysis Summary Described Experiments 1 independent versus joint participant decoding 2 sensitivity to minimum duration constraints 3 sensitivity to maximum overlap constraints 4 generalization to other (non- Bmr ) meetings Evaluation: recall (R), precision (P), and unweighted F goal here: L versus ¬L = S ∪ N sanity: S versus ¬S = L ∪ N sanity: V = S ∪ L versus ¬V = N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  34. Introduction Data Model Experiments Analysis Summary Described Experiments 1 independent versus joint participant decoding 2 sensitivity to minimum duration constraints 3 sensitivity to maximum overlap constraints 4 generalization to other (non- Bmr ) meetings Evaluation: recall (R), precision (P), and unweighted F goal here: L versus ¬L = S ∪ N sanity: S versus ¬S = L ∪ N sanity: V = S ∪ L versus ¬V = N K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  35. Introduction Data Model Experiments Analysis Summary Single-participant vs Multiparticipant Decoding for decoding participants independently N all and N farfield both represent nearfield silence N → 3 competing models, rather than 4 for decoding participant jointly, can use either 3 or 4 models V S L Decoding F R P F R P F indep, 3 AM 76.3 90.3 85.0 87.6 80.9 20.4 32.6 joint, 3 AM 78.8 89.7 86.0 87.8 59.2 20.6 30.6 � joint, 4 AM 83.6 90.0 86.7 55.2 79.5 25.1 34.5 1 joint decoding improves precision by reducing potential overlap 2 modeling farfield vocalization on CTMs significantly improves precision for S and L K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  36. Introduction Data Model Experiments Analysis Summary Single-participant vs Multiparticipant Decoding for decoding participants independently N all and N farfield both represent nearfield silence N → 3 competing models, rather than 4 for decoding participant jointly, can use either 3 or 4 models V S L Decoding F R P F R P F indep, 3 AM 76.3 90.3 85.0 87.6 80.9 20.4 32.6 joint, 3 AM 78.8 89.7 86.0 87.8 59.2 20.6 30.6 � joint, 4 AM 83.6 90.0 86.7 55.2 79.5 25.1 34.5 1 joint decoding improves precision by reducing potential overlap 2 modeling farfield vocalization on CTMs significantly improves precision for S and L K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  37. Introduction Data Model Experiments Analysis Summary Single-participant vs Multiparticipant Decoding for decoding participants independently N all and N farfield both represent nearfield silence N → 3 competing models, rather than 4 for decoding participant jointly, can use either 3 or 4 models V S L Decoding F R P F R P F indep, 3 AM 76.3 90.3 85.0 87.6 80.9 20.4 32.6 joint, 3 AM 78.8 89.7 86.0 87.8 59.2 20.6 30.6 � joint, 4 AM 83.6 90.0 86.7 55.2 79.5 25.1 34.5 1 joint decoding improves precision by reducing potential overlap 2 modeling farfield vocalization on CTMs significantly improves precision for S and L K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  38. Introduction Data Model Experiments Analysis Summary Alternative Minimum Duration Constraints T min T min = (0 . 1 , 0 . 1 , 0 . 1) T min = (0 . 3 , 0 . 3 , 0 . 3) T min = (0 . 2 , 0 . 4 , 0 . 3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  39. Introduction Data Model Experiments Analysis Summary Alternative Minimum Duration Constraints T min L (+1) N (0) S (+1) T min = (0 . 1 , 0 . 1 , 0 . 1) T min = (0 . 3 , 0 . 3 , 0 . 3) T min = (0 . 2 , 0 . 4 , 0 . 3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  40. Introduction Data Model Experiments Analysis Summary Alternative Minimum Duration Constraints T min L (+3) L (+1) L (+2) L (+3) N ( − 3) N ( − 2) L (+1) N (0) N ( − 1) N (0) S (+3) S (+1) S (+1) S (+2) S (+2) T min = (0 . 1 , 0 . 1 , 0 . 1) T min = (0 . 3 , 0 . 3 , 0 . 3) T min = (0 . 2 , 0 . 4 , 0 . 3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  41. Introduction Data Model Experiments Analysis Summary Alternative Minimum Duration Constraints T min L (+3) L (+3) L (+1) L (+2) L (+3) L (+3) L (+2) L (+4) N ( − 3) N ( − 2) L (+1) N (0) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N (0) S (+3) S (+1) N ( − 1) N (0) S (+2) S (+1) S (+1) S (+2) S (+4) S (+2) S (+2) T min = (0 . 1 , 0 . 1 , 0 . 1) T min = (0 . 3 , 0 . 3 , 0 . 3) T min = (0 . 2 , 0 . 4 , 0 . 3) S (+3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  42. Introduction Data Model Experiments Analysis Summary Alternative Minimum Duration Constraints T min hold maximum overlap constraints fixed, K max = (2 , 3 , 3) V S L T min (s) F R P F R P F (0 . 1 , 0 . 1 , 0 . 1) 78.1 82.3 89.9 86.0 55.9 22.1 31.7 (0 . 3 , 0 . 3 , 0 . 3) 79.5 83.7 90.4 86.9 54.7 24.2 33.6 � (0 . 2 , 0 . 4 , 0 . 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing all T min from 0.1s to 0.3s improves all F measures 2 allowing T L min > T S min can result in higher F ( L ) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  43. Introduction Data Model Experiments Analysis Summary Alternative Minimum Duration Constraints T min hold maximum overlap constraints fixed, K max = (2 , 3 , 3) V S L T min (s) F R P F R P F (0 . 1 , 0 . 1 , 0 . 1) 78.1 82.3 89.9 86.0 55.9 22.1 31.7 (0 . 3 , 0 . 3 , 0 . 3) 79.5 83.7 90.4 86.9 54.7 24.2 33.6 � (0 . 2 , 0 . 4 , 0 . 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing all T min from 0.1s to 0.3s improves all F measures 2 allowing T L min > T S min can result in higher F ( L ) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  44. Introduction Data Model Experiments Analysis Summary Alternative Minimum Duration Constraints T min hold maximum overlap constraints fixed, K max = (2 , 3 , 3) V S L T min (s) F R P F R P F (0 . 1 , 0 . 1 , 0 . 1) 78.1 82.3 89.9 86.0 55.9 22.1 31.7 (0 . 3 , 0 . 3 , 0 . 3) 79.5 83.7 90.4 86.9 54.7 24.2 33.6 � (0 . 2 , 0 . 4 , 0 . 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing all T min from 0.1s to 0.3s improves all F measures 2 allowing T L min > T S min can result in higher F ( L ) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  45. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max K max = (3 , 2 , 3) K max = (2 , 2 , 2) K max = (2 , 2 , 3) K max = (2 , 3 , 3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  46. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max L (+3) L (+3) ≤ 2 L (+2) K max = (3 , 2 , 3) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N (0) S (+2) ≤ 2 ≥ ( K − 2) S (+1) S (+4 K max = (2 , 2 , 2) K max = (2 , 2 , 3) S (+2) S (+3) K max = (2 , 3 , 3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  47. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max L (+3) L (+3) L (+3) L (+3) ≤ 2 ≤ 2 L (+2) L (+2) K max = (3 , 2 , 3) L (+4) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) ≤ 2 ≤ 2 ≥ ( K − 2) ≥ ( K − 3) S (+1) S (+1) S (+4 S (+4 K max = (2 , 2 , 2) K max = (2 , 2 , 3) S (+2) S (+2) S (+3) S (+3) K max = (2 , 3 , 3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  48. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max L (+3) L (+3) ≤ 2 L (+2) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 1) L (+3) L (+3) N ( − 1) N (0) S (+2) ≤ 3 ≥ ( K − 3) S (+1) L (+3) L (+3) ≤ 2 ≤ 2 S (+4 L (+2) L (+2) K max = (3 , 2 , 3) S (+2) L (+4) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) S (+3) N ( − 1) N ( − 1) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) ≤ 2 ≤ 2 ≥ ( K − 2) ≥ ( K − 3) S (+1) S (+1) S (+4 S (+4 K max = (2 , 2 , 2) K max = (2 , 2 , 3) S (+2) S (+2) S (+3) S (+3) K max = (2 , 3 , 3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  49. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max L (+3) L (+3) ≤ 2 L (+2) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 1) L (+3) L (+3) N ( − 1) N (0) S (+2) ≤ 3 ≥ ( K − 3) S (+1) L (+3) L (+3) ≤ 2 ≤ 2 S (+4 L (+2) L (+2) K max = (3 , 2 , 3) S (+2) L (+4) L (+4) N ( − 3) N ( − 2) L (+1) N ( − 3) N ( − 2) L (+1) S (+3) N ( − 1) N ( − 1) N ( − 1) N (0) S (+2) N ( − 1) N (0) S (+2) L (+3) ≤ 2 ≤ 2 ≥ ( K − 2) ≥ ( K − 3) S (+1) S (+1) L (+3) ≤ 3 S (+4 S (+4 L (+2) K max = (2 , 2 , 2) K max = (2 , 2 , 3) S (+2) S (+2) L (+4) S (+3) S (+3) N ( − 3) N ( − 2) L (+1) N ( − 1) N ( − 1) N (0) S (+2) ≤ 2 ≥ ( K − 3) S (+1) S (+4 K max = (2 , 3 , 3) S (+2) S (+3) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  50. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max minimum duration constraints fixed, T min = (0 . 2 , 0 . 4 , 0 . 3) V S L K max F R P F R P F (2 , 2 , 2) 81.3 83.3 90.6 86.8 36.9 27.8 31.7 (2 , 2 , 3) 79.9 84.0 89.0 86.4 48.8 24.3 32.4 (3 , 2 , 3) 79.9 84.2 88.6 86.4 49.1 24.6 32.8 � (2 , 3 , 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing K max generally leads to higher R and lower P 2 increasing K S max from 2 to 3 has negligible impact 3 increasing K L max from 2 to 3 has significant impact ⋆ because a higher proportion of L is produced in overlap K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  51. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max minimum duration constraints fixed, T min = (0 . 2 , 0 . 4 , 0 . 3) V S L K max F R P F R P F (2 , 2 , 2) 81.3 83.3 90.6 86.8 36.9 27.8 31.7 (2 , 2 , 3) 79.9 84.0 89.0 86.4 48.8 24.3 32.4 (3 , 2 , 3) 79.9 84.2 88.6 86.4 49.1 24.6 32.8 � (2 , 3 , 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing K max generally leads to higher R and lower P 2 increasing K S max from 2 to 3 has negligible impact 3 increasing K L max from 2 to 3 has significant impact ⋆ because a higher proportion of L is produced in overlap K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  52. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max minimum duration constraints fixed, T min = (0 . 2 , 0 . 4 , 0 . 3) V S L K max F R P F R P F (2 , 2 , 2) 81.3 83.3 90.6 86.8 36.9 27.8 31.7 (2 , 2 , 3) 79.9 84.0 89.0 86.4 48.8 24.3 32.4 (3 , 2 , 3) 79.9 84.2 88.6 86.4 49.1 24.6 32.8 � (2 , 3 , 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing K max generally leads to higher R and lower P 2 increasing K S max from 2 to 3 has negligible impact 3 increasing K L max from 2 to 3 has significant impact ⋆ because a higher proportion of L is produced in overlap K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  53. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max minimum duration constraints fixed, T min = (0 . 2 , 0 . 4 , 0 . 3) V S L K max F R P F R P F (2 , 2 , 2) 81.3 83.3 90.6 86.8 36.9 27.8 31.7 (2 , 2 , 3) 79.9 84.0 89.0 86.4 48.8 24.3 32.4 (3 , 2 , 3) 79.9 84.2 88.6 86.4 49.1 24.6 32.8 � (2 , 3 , 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing K max generally leads to higher R and lower P 2 increasing K S max from 2 to 3 has negligible impact 3 increasing K L max from 2 to 3 has significant impact ⋆ because a higher proportion of L is produced in overlap K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  54. Introduction Data Model Experiments Analysis Summary Alternative Maximum Overlap Constraints K max minimum duration constraints fixed, T min = (0 . 2 , 0 . 4 , 0 . 3) V S L K max F R P F R P F (2 , 2 , 2) 81.3 83.3 90.6 86.8 36.9 27.8 31.7 (2 , 2 , 3) 79.9 84.0 89.0 86.4 48.8 24.3 32.4 (3 , 2 , 3) 79.9 84.2 88.6 86.4 49.1 24.6 32.8 � (2 , 3 , 3) 79.5 83.6 90.0 86.7 55.2 25.1 34.5 1 increasing K max generally leads to higher R and lower P 2 increasing K S max from 2 to 3 has negligible impact 3 increasing K L max from 2 to 3 has significant impact ⋆ because a higher proportion of L is produced in overlap K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  55. Introduction Data Model Experiments Analysis Summary Generalization to Other Meetings V S L Test data p V ( L ) F R P F R P F train 10.91 80.1 83.4 89.8 86.5 53.0 19.4 28.4 Bmr test 14.94 79.5 83.6 90.0 86.7 55.2 25.1 34.5 (all) 5.94 78.1 81.1 85.6 57.8 11.4 19.0 Bro 90.6 (all) 7.53 75.1 85.7 85.2 10.0 17.0 Bed 84.6 58.7 1 F ( V ): Bmr (train) > Bmr (test) > Bro > Bed Bmr (train) and Bmr (test) have lots of participants in common with Bmr , Bro shares 3 participants, and Bed 1 participant 2 F ( L ) on Bmr (test) higher than on Bmr (train) appears to correlate with p V ( L ), the proportion of vocalization effort spent on laughter this test set is not typical of the corpus K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  56. Introduction Data Model Experiments Analysis Summary Generalization to Other Meetings V S L Test data p V ( L ) F R P F R P F train 10.91 80.1 83.4 89.8 86.5 53.0 19.4 28.4 Bmr test 14.94 79.5 83.6 90.0 86.7 55.2 25.1 34.5 (all) 5.94 78.1 81.1 85.6 57.8 11.4 19.0 Bro 90.6 (all) 7.53 75.1 85.7 85.2 10.0 17.0 Bed 84.6 58.7 1 F ( V ): Bmr (train) > Bmr (test) > Bro > Bed Bmr (train) and Bmr (test) have lots of participants in common with Bmr , Bro shares 3 participants, and Bed 1 participant 2 F ( L ) on Bmr (test) higher than on Bmr (train) appears to correlate with p V ( L ), the proportion of vocalization effort spent on laughter this test set is not typical of the corpus K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  57. Introduction Data Model Experiments Analysis Summary Generalization to Other Meetings V S L Test data p V ( L ) F R P F R P F train 10.91 80.1 83.4 89.8 86.5 53.0 19.4 28.4 Bmr test 14.94 79.5 83.6 90.0 86.7 55.2 25.1 34.5 (all) 5.94 78.1 81.1 85.6 57.8 11.4 19.0 Bro 90.6 (all) 7.53 75.1 85.7 85.2 10.0 17.0 Bed 84.6 58.7 1 F ( V ): Bmr (train) > Bmr (test) > Bro > Bed Bmr (train) and Bmr (test) have lots of participants in common with Bmr , Bro shares 3 participants, and Bed 1 participant 2 F ( L ) on Bmr (test) higher than on Bmr (train) appears to correlate with p V ( L ), the proportion of vocalization effort spent on laughter this test set is not typical of the corpus K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  58. Introduction Data Model Experiments Analysis Summary Generalization to Other Meetings V S L Test data p V ( L ) F R P F R P F train 10.91 80.1 83.4 89.8 86.5 53.0 19.4 28.4 Bmr test 14.94 79.5 83.6 90.0 86.7 55.2 25.1 34.5 (all) 5.94 78.1 81.1 85.6 57.8 11.4 19.0 Bro 90.6 (all) 7.53 75.1 85.7 85.2 10.0 17.0 Bed 84.6 58.7 1 F ( V ): Bmr (train) > Bmr (test) > Bro > Bed Bmr (train) and Bmr (test) have lots of participants in common with Bmr , Bro shares 3 participants, and Bed 1 participant 2 F ( L ) on Bmr (test) higher than on Bmr (train) appears to correlate with p V ( L ), the proportion of vocalization effort spent on laughter this test set is not typical of the corpus K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  59. Introduction Data Model Experiments Analysis Summary Generalization to Other Meetings V S L Test data p V ( L ) F R P F R P F train 10.91 80.1 83.4 89.8 86.5 53.0 19.4 28.4 Bmr test 14.94 79.5 83.6 90.0 86.7 55.2 25.1 34.5 (all) 5.94 78.1 81.1 85.6 57.8 11.4 19.0 Bro 90.6 (all) 7.53 75.1 85.7 85.2 10.0 17.0 Bed 84.6 58.7 1 F ( V ): Bmr (train) > Bmr (test) > Bro > Bed Bmr (train) and Bmr (test) have lots of participants in common with Bmr , Bro shares 3 participants, and Bed 1 participant 2 F ( L ) on Bmr (test) higher than on Bmr (train) appears to correlate with p V ( L ), the proportion of vocalization effort spent on laughter this test set is not typical of the corpus K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  60. Introduction Data Model Experiments Analysis Summary Generalization to Other Meetings V S L Test data p V ( L ) F R P F R P F train 10.91 80.1 83.4 89.8 86.5 53.0 19.4 28.4 Bmr test 14.94 79.5 83.6 90.0 86.7 55.2 25.1 34.5 (all) 5.94 78.1 81.1 85.6 57.8 11.4 19.0 Bro 90.6 (all) 7.53 75.1 85.7 85.2 10.0 17.0 Bed 84.6 58.7 1 F ( V ): Bmr (train) > Bmr (test) > Bro > Bed Bmr (train) and Bmr (test) have lots of participants in common with Bmr , Bro shares 3 participants, and Bed 1 participant 2 F ( L ) on Bmr (test) higher than on Bmr (train) appears to correlate with p V ( L ), the proportion of vocalization effort spent on laughter this test set is not typical of the corpus K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  61. Introduction Data Model Experiments Analysis Summary Generalization to Other Meetings V S L Test data p V ( L ) F R P F R P F train 10.91 80.1 83.4 89.8 86.5 53.0 19.4 28.4 Bmr test 14.94 79.5 83.6 90.0 86.7 55.2 25.1 34.5 (all) 5.94 78.1 81.1 85.6 57.8 11.4 19.0 Bro 90.6 (all) 7.53 75.1 85.7 85.2 10.0 17.0 Bed 84.6 58.7 1 F ( V ): Bmr (train) > Bmr (test) > Bro > Bed Bmr (train) and Bmr (test) have lots of participants in common with Bmr , Bro shares 3 participants, and Bed 1 participant 2 F ( L ) on Bmr (test) higher than on Bmr (train) appears to correlate with p V ( L ), the proportion of vocalization effort spent on laughter this test set is not typical of the corpus K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  62. Introduction Data Model Experiments Analysis Summary Confusion Matrix Analysis hypothesized as Σ N L S N 22.9 7.8 716.2 685.4 L 6.5 9.1 1.0 10.4 S 11.0 4.5 79.0 94.4 Σ 702.9 36.6 87.8 827.2 final system on test set (13.8 hours) all quantities in minutes K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  63. Introduction Data Model Experiments Analysis Summary Confusion Matrix Analysis hypothesized as Σ N L S N 22.9 7.8 716.2 685.4 L 6.5 9.1 1.0 16.6 S 11.0 4.5 79.0 94.4 Σ 702.9 36.6 87.8 827.2 break down references L ≡ { L ′ U , L ′ V , L ∩ S } L ′ U ≡ L U − L ∩ S : unvoiced laughter less “laughed speech” L ′ V ≡ L V − L ∩ S : voiced laughter less “laughed speech” K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  64. Introduction Data Model Experiments Analysis Summary Confusion Matrix Analysis hypothesized as Σ N L S N 22.9 7.8 716.2 685.4 L 6.5 9.1 1.0 16.6 S 11.0 4.5 79.0 94.4 Σ 702.9 36.6 87.8 827.2 break down references L ≡ { L ′ U , L ′ V , L ∩ S } L ′ U ≡ L U − L ∩ S : unvoiced laughter less “laughed speech” L ′ V ≡ L V − L ∩ S : voiced laughter less “laughed speech” K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  65. Introduction Data Model Experiments Analysis Summary Confusion Matrix Analysis hypothesized as Σ N L S N 22.9 7.8 716.2 685.4 L 6.5 9.1 1.0 16.6 S 11.0 4.5 79.0 94.4 Σ 702.9 36.6 87.8 827.2 break down references L ≡ { L ′ U , L ′ V , L ∩ S } L ′ U ≡ L U − L ∩ S : unvoiced laughter less “laughed speech” L ′ V ≡ L V − L ∩ S : voiced laughter less “laughed speech” K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  66. Introduction Data Model Experiments Analysis Summary Confusion Matrix Analysis hypothesized as Σ N L S N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 U L ′ 3.6 6.5 0.3 10.4 V L ∩ S 0.1 0.2 0.5 0.8 S 11.0 4.5 79.0 94.4 Σ 702.9 36.6 87.8 827.2 1 most unvoiced laughter ( L ′ U ) is classified as silence ( N ) 2 most “laughed speech” ( L ∩ S ) is classified as speech ( S ) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  67. Introduction Data Model Experiments Analysis Summary Confusion Matrix Analysis hypothesized as Σ N L S N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 U L ′ 3.6 6.5 0.3 10.4 V L ∩ S 0.1 0.2 0.5 0.8 S 11.0 4.5 79.0 94.4 Σ 702.9 36.6 87.8 827.2 1 most unvoiced laughter ( L ′ U ) is classified as silence ( N ) 2 most “laughed speech” ( L ∩ S ) is classified as speech ( S ) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  68. Introduction Data Model Experiments Analysis Summary Confusion Matrix Analysis hypothesized as Σ N L S N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 U L ′ 3.6 6.5 0.3 10.4 V L ∩ S 0.1 0.2 0.5 0.8 S 11.0 4.5 79.0 94.4 Σ 702.9 36.6 87.8 827.2 1 most unvoiced laughter ( L ′ U ) is classified as silence ( N ) 2 most “laughed speech” ( L ∩ S ) is classified as speech ( S ) K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  69. Introduction Data Model Experiments Analysis Summary Confusions Between L and S Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 L S U L ′ L ′ 3.6 0.3 10.4 6.5 8.9 0.5 V L ∩ S 0.1 0.2 0.5 0.8 L ∩ S 0.2 0.5 S 11.0 4.5 94.4 S 79.0 4.5 79.0 Σ 702.9 36.6 87.8 827.2 looking at L and S only K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  70. Introduction Data Model Experiments Analysis Summary Confusions Between L and S Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 L S U L ′ L ′ 3.6 0.3 10.4 6.5 94.7 5.3 V L ∩ S 0.1 0.2 0.5 0.8 L ∩ S 28.6 71.4 S 11.0 4.5 94.4 S 79.0 5.4 93.6 Σ 702.9 36.6 87.8 827.2 1 94% of speech is hypothesized as speech 2 95% of laughter (excluding “laughed speech”) is hypothesized as laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  71. Introduction Data Model Experiments Analysis Summary Confusions Between L and S Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 L S U L ′ L ′ 3.6 0.3 10.4 6.5 94.7 5.3 V L ∩ S 0.1 0.2 0.5 0.8 L ∩ S 28.6 71.4 S 11.0 4.5 94.4 S 79.0 5.4 93.6 Σ 702.9 36.6 87.8 827.2 1 94% of speech is hypothesized as speech 2 95% of laughter (excluding “laughed speech”) is hypothesized as laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  72. Introduction Data Model Experiments Analysis Summary Confusions Between L and S Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 L S U L ′ L ′ 3.6 0.3 10.4 6.5 94.7 5.3 V L ∩ S 0.1 0.2 0.5 0.8 L ∩ S 28.6 71.4 S 11.0 4.5 94.4 S 79.0 5.4 93.6 Σ 702.9 36.6 87.8 827.2 1 94% of speech is hypothesized as speech 2 95% of laughter (excluding “laughed speech”) is hypothesized as laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  73. Introduction Data Model Experiments Analysis Summary Confusions Between L and S Precision: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 L S U L ′ L ′ 3.6 0.3 10.4 6.5 65.4 0.6 V L ∩ S 0.1 0.2 0.5 0.8 L ∩ S 1.5 0.6 S 11.0 4.5 94.4 S 79.0 33.1 98.8 Σ 702.9 36.6 87.8 827.2 1 99% of hypothesized speech is speech 2 65% of hypothesized laughter is laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  74. Introduction Data Model Experiments Analysis Summary Confusions Between L and S Precision: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 L S U L ′ L ′ 3.6 0.3 10.4 6.5 65.4 0.6 V L ∩ S 0.1 0.2 0.5 0.8 L ∩ S 1.5 0.6 S 11.0 4.5 94.4 S 79.0 33.1 98.8 Σ 702.9 36.6 87.8 827.2 1 99% of hypothesized speech is speech 2 65% of hypothesized laughter is laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  75. Introduction Data Model Experiments Analysis Summary Confusions Between L and S Precision: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 L S U L ′ L ′ 3.6 0.3 10.4 6.5 65.4 0.6 V L ∩ S 0.1 0.2 0.5 0.8 L ∩ S 1.5 0.6 S 11.0 4.5 94.4 S 79.0 33.1 98.8 Σ 702.9 36.6 87.8 827.2 1 99% of hypothesized speech is speech 2 65% of hypothesized laughter is laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  76. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 685.4 22.9 V L ′ L ∩ S 0.1 0.2 0.5 0.8 2.8 2.4 U S 11.0 4.5 94.4 L V 79.0 3.7 6.7 Σ 702.9 36.6 87.8 827.2 looking at L and N only K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  77. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 96.8 3.2 V L ′ L ∩ S 0.1 0.2 0.5 0.8 53.9 46.2 U S 11.0 4.5 94.4 L V 79.0 35.6 64.4 Σ 702.9 36.6 87.8 827.2 1 97% of silence is hypothesized as silence 2 64% of voiced laughter (including “laughed speech”) is classified as laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  78. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 96.8 3.2 V L ′ L ∩ S 0.1 0.2 0.5 0.8 53.9 46.2 U S 11.0 4.5 94.4 L V 79.0 35.6 64.4 Σ 702.9 36.6 87.8 827.2 1 97% of silence is hypothesized as silence 2 64% of voiced laughter (including “laughed speech”) is classified as laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  79. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Recall: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 96.8 3.2 V L ′ L ∩ S 0.1 0.2 0.5 0.8 53.9 46.2 U S 11.0 4.5 94.4 L V 79.0 35.6 64.4 Σ 702.9 36.6 87.8 827.2 1 97% of silence is hypothesized as silence 2 64% of voiced laughter (including “laughed speech”) is classified as laughter K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  80. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Precision: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 99.1 71.6 V L ′ L ∩ S 0.1 0.2 0.5 0.8 0.4 7.5 U S 11.0 4.5 94.4 L V 79.0 0.5 20.9 Σ 702.9 36.6 87.8 827.2 1 99% of hypothesized silence is silence 2 28% of hypothesized laughter is laughter 3 72% of hypothesized laughter is silence K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  81. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Precision: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 99.1 71.6 V L ′ L ∩ S 0.1 0.2 0.5 0.8 0.4 7.5 U S 11.0 4.5 94.4 L V 79.0 0.5 20.9 Σ 702.9 36.6 87.8 827.2 1 99% of hypothesized silence is silence 2 28% of hypothesized laughter is laughter 3 72% of hypothesized laughter is silence K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  82. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Precision: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 99.1 71.6 V L ′ L ∩ S 0.1 0.2 0.5 0.8 0.4 7.5 U S 11.0 4.5 94.4 L V 79.0 0.5 20.9 Σ 702.9 36.6 87.8 827.2 1 99% of hypothesized silence is silence 2 28% of hypothesized laughter is laughter 3 72% of hypothesized laughter is silence K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

  83. Introduction Data Model Experiments Analysis Summary Confusions Between L and N Precision: N L S Σ N 22.9 7.8 716.2 685.4 L ′ 2.8 2.4 0.2 5.4 N L U L ′ 3.6 0.3 10.4 N 6.5 99.1 71.6 V L ′ L ∩ S 0.1 0.2 0.5 0.8 0.4 7.5 U S 11.0 4.5 94.4 L V 79.0 0.5 20.9 Σ 702.9 36.6 87.8 827.2 1 99% of hypothesized silence is silence 2 28% of hypothesized laughter is laughter 3 72% of hypothesized laughter is silence K. Laskowski & T. Schultz Detection of Laughter-in-Interaction in Meetings

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend