Measuring Adaptation Between Dialogs - - PowerPoint PPT Presentation

• Measuring Adaptation

Between Dialogs

• Adaptation in Dialog

!"#$###

#"%%#&''#' (' )*"'$' !"&#'#

#'

• +

Examples of Lexical Variation

,-. ,'$. ,(%''. ,$. ,". ,#'. ,/#'. ,!$.

• Examples of Lexical Variation

,-. ,'$. ,(%''. ,$.

• 1

Examples of Syntactic Variation

• ,)"2 3.

,)"32 .

• 4

Evidence of Adaptation in Dialog

5#%&#'6

,&###2###'#"2

'#'###'# ##" '#.

/%#"&''#' ##'

! "#

• Causes of Adaptation

/

7$%#& !%'(() %*

(#

%+!%'(('

• '#'#"2$

''###"

• 8

Recency

9'$#""$"

$#

'6"##"

#"

• 7 ':';8<

,%$#$$ #"' '#$'$'## :''$.

• ;

Partner Adaptation

'% '6#$#'#"

,"2$ %#'2$#=.> ''6

$' 2''##$'

'''

• Corpus Studies On Recency

Adaptation

?!$@'$&##

,##$. #%# '

?$2 >>4@##''$

'$'####"' #&

?/# > 4@'$#''*

##"%%#3' $'#""#'# "''#

$ ##%

*'#%# #

• Setup

+' '6!

>

*A #'#2

>

*A! ' %%

+>

*A ' %%

• !##
• !#+#

• Maptask Corpus Structure

2 2 8

• 2

4

• 2
• 1
• 2

2

• +

#

• 2
• #

2

• Hypothesis:

recency adaptation happens between (1-4) and (2-3) partner adaptation happens between (1-6) and (2-5)

• +

Church’s measure for adaptation

• Church’s measure for adaptation

9#'''%$$#%

$'>-'$#'#2 %,". ''

)#"22##%$',.,.

$ #'$'

B(#B('##B #%

• 1

Proposed Adaptation Measure

• * '$'

#>

C

'#'%#2 %$'2#"#'

C

#2'''

• C

'#"%D$## %%'%D$#"

! #%%$#'# $%D$%# 'E#F #$##"

• 4

Proposed Adaptation Measures

• Terminology

'$%6GD$%%$%# $#'"2'# " "%D$% %$%#$' #%#$'## #"#%D$"2'# ' #"

• 8

Adaptation Ratio

P T P∩T

Chance: probability that f co-occurs in prime and target by chance:

N – total number of (prime, target) dialog pairs P – number of prime dialogs where freq of f > b T – number of target dialogs where freq of f > b

ƒєPrime ƒєTarget ƒєPrime and ƒє Target All Dialog pairs

N

• ;

Adaptation Ratio

Adaptation Ratio = +adapt / chance

Positive Adaptation:

P T

P∩T

N – total number of (prime, target) dialog pairs P – number of prime dialogs where freq of f > b T – number of target dialogs where freq of f > b

ƒєPrime

ƒєTarget

ƒєPrime and ƒє Target

N - All Dialog pairs

Chance probability that f co-

• ccurs in prime and target by

chance

• Adaptation Ratio

Adaptation Ratio = +adapt / chance

Positive Adaptation:

P T P∩T

N – total number of (prime, target) dialog pairs P – number of prime dialogs where freq of f > b T – number of target dialogs where freq of f > b

ƒєPrime ƒєTarget ƒєPrime and ƒє Target N - All Dialog pairs

• Adaptation Ratio

'$'#

C

'#'%# 2#%%%$'

C

#2%$'%#$' %D$#'

! #%%$#'# $%D$%# 'E#F #$##"

• Distance Measure

C '#"##"%%'

%D$#"6 %%$#'# #'#'&%D$# "H

• +

Distance Measure

For a feature f in a dialog pair (Prime, Target): t – frequency of feature f in target dialog p – frequency of feature f in prime dialog b – average frequency of f Feature f is “adapted” in a pair of dialogs if distance >0 Strength of adaptation is proportional to the distance

• Experimental Questions

>

#%%$'&#2# #> !2$'H

>

9#%##' 6H

+>

'%$%D$# ##"%%%$ %D$#"H

• 1

Features

9'#(I"'J

#'#"$#'2'''

#"' '#(I"' #7%3'

#'<

• 4

Word-Stems with Adaptation Ratio > 1 and significant

• 2

I ' " '' K5/ %2 ''$" "% (/5( $ #

• L

#M$' $''

• 5-

#' #% !IL #"2$ #" K 2#"* #"* L

• Only features occuring in > 30% of prime dialogs with freq > baseline

Relative direction Contains ‘you’ Contains ‘I’ or ‘my

• Bigrams with Adaptation Ratio > 1 and

significant

your left, right-hand side, come to, you come, about the, when you, go round, and round, you got, if you, up toward, a wee, you just, round the, right you, just abov, abov the no no, my map,

• kay and,

you just,

• n my,

down about, yeah i, you got, down to, have a, i mean, ’til you, just below, just to, now you, no you

Partner Recency

Relative direction Contains ‘you’ Contains ‘I’ or ‘my

Only features occuring in > 30% of prime dialogs with freq > baseline

• 8

Comparing Partner and Recency Adaptation

2.4 2.5 2.6 2.7 2.8 2.9 3 3.1 Word Bigram Syntactic Partner Recency

adaptation ratio and adaptation strength are averaged over all features for each feature type

Adaptation Ratio = +adapt / chance

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Word Bigram Syntactic Partner Recency

Distance measure ~ Adaptation strength

* * * * * Indicates significance (p<.05)

• !

"#$

$%&'(&'(()

• ;

Adaptation Ratio for Selected Syntactic Features

0.5 1 1.5 2 2.5 3 3.5 NP->NP PP NP->NN NP->DT NN NP->DT AP NN Partner Recency

The features examined by Dubey et.al 2006 on Switchboard corpus Found adaptation between speakers

• +

Adaptation for Selected Syntactic Features

5 10 15 20 25 30 35 NP->NP PP NP->NN NP->DT NN NP->DT AP NN Partner Recency

NP->NN PP rule: adaptation ratio is stronger for recency means that if primed, speaker is more likely to use this rule in the very next conversation Adaptation Strength is higher for the partner scenario means that the “adapting speaker” in partner-scenario will use this rule with higher frequency than the “adapting speaker” in recency-scenario

0.5 1 1.5 2 2.5 3 3.5 NP->NP PP NP->NN NP->DT NN NP->DT AP NN Partner Recency

Adaptation Ratio = +adapt / chance Distance measure ~ Adaptation strength

• +

Dubey’s Results

• +

Effect of Frequency in the Prime

0.5 1 1.5 2 2.5 3 3.5 4 4.5 prime>base prime>base+1 prime>base+2 Partner Recency

#"

0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 prime>base prime>base+1 prime>base+2 Partner Recency

N%%$' !$##%%$' !'#%$2,. #%#/#A

GD$%%$## '%%%# $%%''"%#

• ++

Conclusions

#%#'%%$'%%#"

##%%#$$'

G$#%%

#2#"'

###''"#

'#

###

'"'#

• +0

Implications

5#%# '$""''2%#'%

• $#"'#"'7$

2$22###'<%#" ''#$$'

#"#%#2

'#""$""# $'

• +1

Future Work

9'$""'#'%

'$'$ #%%#2 #>

• +4

Future Work

'#'#*%D$

'

!'##%E F 3'$##*#"# !'#'$###"

#"%'#6

<*A<*A <*A<*A!

• #$

'#' >

• #$"=##

• +

O$'#'H

''# P"#>

• $