Modeling Meeting Turns Dan Ellis <dpwe@ee.columbia.edu> - PowerPoint PPT Presentation

Modeling Meeting Turns Dan Ellis <dpwe@ee.columbia.edu> LabROSA, Columbia University & ICSI • Meeting turns visualization • Turn-pattern segmentation • ‘Talkativity’ modeling m4 meeting - Dan Ellis 2003-01-29

Meeting Turn Visualization • Speaker turns form patterns on multi- minute timescales: mr04: Hand-marked speaker turns 0: 9: 8: 7: 5: 3: 2: 1: 0 5 10 15 20 25 30 35 40 45 50 55 60 time / minutes • Points of pattern change are ‘significant’? topics? modes? m4 meeting - Dan Ellis 2003-01-29

Modeling meeting segments • Model speaker activity patterns like states 15 20 25 time / min • Prior vector: 0.3 Prob 0.2 P ( spkr i ) 0.1 0 0 1 2 3 4 5 6 7 spkr spkr(t) 7 • ‘Transition’ matrix: 6 5 4 3 P ( spkr i t , spkr j t-1 ) 2 1 0 0 1 2 3 4 5 6 7 spkr(t+1) m4 meeting - Dan Ellis 2003-01-29

Self-similarity • Display Dist ( minute i , minute j ) as KL distance of speaker distributions mr04: Self-sim of turn mxs by KL time/min 60 8 50 40 6 30 4 20 2 10 KL dist 20 40 60 time/min m4 meeting - Dan Ellis 2003-01-29

BIC Segmentation • BIC (Bayesian Information Criterion): Compare more and less complex models log L ( X 1 ; M 1 ) L ( X 2 ; M 2 ) ≷ λ 2 log( N ) ∆ #( M ) L ( X ; M 0 ) • For segmentation: Grow context window from current boundary For each window, test every possible segmentation When BIC is positive, mark new segment current candidate last context limit boundary segmentation point 0 N time L ( X 2 ; M 2 ) L ( X 1 ; M 1 ) L ( X ; M 0 ) m4 meeting - Dan Ellis 2003-01-29

BIC Segmentation • Example of boundary finding: 8 7 Participants 6 5 4 3 2 1 15 16 17 18 19 20 21 22 23 24 25 boundary 0 passes BIC last BIC score seg point current -100 context limit no boundary found -200 with shorter context 15 16 17 18 19 20 21 22 23 24 25 time / min m4 meeting - Dan Ellis 2003-01-29

BIC Segmentation • Appears to find shifts in turn patterns: mr04: Hand-marked speaker turns vs. time + auto/manual boundaries 0: 9: 8: 7: 5: 3: 2: 1: 0 5 10 15 20 25 30 35 40 45 50 55 60 • Evaluate against topic boundaries time/min (6 meetings, 36 boundaries) 15 (42%) agree to within ± 2 minutes 16 ‘false alarm’ insertions m4 meeting - Dan Ellis 2003-01-29

“Talkativity” • Factors affecting how much one person speaks in a given meeting: indexable relevance/interest of topic to speaker confounding competition with other speakers innate tendency to talk - “talkativity” T s • Model of expected ‘airtime’ consumed by each participant s in meeting m : T s P sm = � t ∈ Sm T t given { T s }, deviations from expected values factor out competition, baseline talkativity m4 meeting - Dan Ellis 2003-01-29

Estimating “Talkativity” • Find best-fitting { T s } to fit meeting set � P sm t ∈ Sm,t � = s T t T s = avg m ∈ M s 1 − P sm Iteratively recalculate { T s } until (fast) convergence 26 meetings (mr* set), 10 common participants, avg 6.9 participants/meeting • Calculate actual:predicted ratios m4 meeting - Dan Ellis 2003-01-29

“Talkativity” Results • Meeting proportions & ratio to prediction Talkativity index Proportion of meeting time per participant 10 9 0.4 8 7 Participant 0.3 6 5 0.2 4 3 0.1 2 0 1 0 0.1 0.2 2 4 6 8 10 12 14 16 18 20 22 24 26 Meeting proportions: log 2 (actual/predicted) 10 1 9 8 0.5 7 Participant 6 0 5 4 -0.5 3 2 -1 1 2 4 6 8 10 12 14 16 18 20 22 24 26 Evaluation? Meeting number m4 meeting - Dan Ellis 2003-01-29