Modeling Meeting Turns Dan Ellis <dpwe@ee.columbia.edu> - - PowerPoint PPT Presentation

m4 meeting - Dan Ellis 2003-01-29

• Meeting turns visualization
• Turn-pattern segmentation
• ‘Talkativity’ modeling

Modeling Meeting Turns

Dan Ellis <dpwe@ee.columbia.edu> LabROSA, Columbia University & ICSI

m4 meeting - Dan Ellis 2003-01-29

• Speaker turns form patterns on multi-

minute timescales:

• Points of pattern change are ‘significant’?

topics? modes?

Meeting Turn Visualization

3: 8: 0: 9: 7: 5: 2: 1: 5 10 15 20 25 30 35 40 45 50 55 60 mr04: Hand-marked speaker turns time / minutes

m4 meeting - Dan Ellis 2003-01-29

• Model speaker activity patterns like states
• Prior vector:

P(spkri)

• ‘Transition’ matrix:

P(spkri

t,spkrj t-1)

Modeling meeting segments

15 20 25 time / min

1 2 3 4 5 6 7 0.1 0.2 0.3 spkr Prob 1 2 3 4 5 6 7 1 2 3 4 5 6 7 spkr(t+1) spkr(t)

m4 meeting - Dan Ellis 2003-01-29

• Display Dist(minutei, minutej)

as KL distance of speaker distributions

Self-similarity

20 40 60 10 20 30 40 50 60 2 4 6 8

time/min time/min KL dist mr04: Self-sim of turn mxs by KL

m4 meeting - Dan Ellis 2003-01-29

• BIC (Bayesian Information Criterion):

Compare more and less complex models

• For segmentation:

Grow context window from current boundary For each window, test every possible segmentation When BIC is positive, mark new segment

BIC Segmentation

L(X;M0)

last segmentation point current context limit candidate boundary

L(X1;M1) L(X2;M2)

time N

log L(X1;M1)L(X2;M2)

L(X;M0)

≷ λ

2 log(N)∆#(M)

m4 meeting - Dan Ellis 2003-01-29

• Example of boundary finding:

BIC Segmentation

15 16 17 18 19 20 21 22 23 24 25 2 4 6 8 1 3 5 7 15 16 17 18 19 20 21 22 23 24 25

• 200
• 100

time / min BIC score Participants

last seg point current context limit boundary passes BIC no boundary found with shorter context

m4 meeting - Dan Ellis 2003-01-29

• Appears to find shifts in turn patterns:
• Evaluate against topic boundaries

(6 meetings, 36 boundaries)

15 (42%) agree to within ± 2 minutes 16 ‘false alarm’ insertions

BIC Segmentation

3: 8: 0: 9: 7: 5: 2: 1: 5 10 15 20 25 30 35 40 45 50 55 60 mr04: Hand-marked speaker turns vs. time + auto/manual boundaries time/min

m4 meeting - Dan Ellis 2003-01-29

• Factors affecting how much one person

speaks in a given meeting:

relevance/interest of topic to speaker competition with other speakers innate tendency to talk - “talkativity” Ts

• Model of expected ‘airtime’ consumed by

each participant s in meeting m:

given {Ts}, deviations from expected values factor out competition, baseline talkativity

“Talkativity”

Psm =

Ts

• t∈Sm Tt

indexable confounding

m4 meeting - Dan Ellis 2003-01-29

• Find best-fitting {Ts} to fit meeting set

Iteratively recalculate {Ts} until (fast) convergence 26 meetings (mr* set), 10 common participants, avg 6.9 participants/meeting

• Calculate actual:predicted ratios

Estimating “Talkativity”

Ts = avgm∈Ms

Psm

• t∈Sm,t=s Tt

1−Psm

m4 meeting - Dan Ellis 2003-01-29

• Meeting proportions & ratio to prediction

“Talkativity” Results

Evaluation?

Talkativity index Participant Proportion of meeting time per participant 2 4 6 8 10 12 14 16 18 20 22 24 26 1 0.1 0.2 2 3 4 5 6 7 8 9 10 Participant 1 2 3 4 5 6 7 8 9 10 Meeting number Meeting proportions: log2(actual/predicted) 2 4 6 8 10 12 14 16 18 20 22 24 26

• 1
• 0.5

0.5 1 0.1 0.2 0.3 0.4