Dialogue Summarization Presenter: Wang Chen Mentor: Piji Li 1 - - PowerPoint PPT Presentation
Dialogue Summarization Presenter: Wang Chen Mentor: Piji Li 1 Outline Introduction Task Definition & Applications Taxonomy Based on Data Type Challenges Recent Work Keep Meeting Summaries on Topic: Abstractive
Presenter: Wang Chen Mentor: Piji Li
1
2019.
2
summary to capture the highlights of the dialogue.
SAMSum [4] Dial2Desc [3] DiDi Customer Service [2]
3
DiDi Customer Service [6] AMI, Meeting Summarization [7] Medical Conversation Summarization [8]
4
Dialogue Summarization Video & Audio Audio Only Text Only
Encoder [3]
5
6
Dialogue Summarization Video & Audio Audio Only Text Only
Encoder [3]
7
topic segment, utterance, and word.
to help recognize the important utterances.
8
important if its speaker receives more attention.
The color indicates the attention received by the speaker PM (Project Manager). Others: ME (Marketing Expert), ID (Industrial Designer), UI (User Interface).
9
We formulate a meeting transcript as a list of triples 𝑌={(𝑞𝑗; 𝑔
𝑗; 𝑣𝑗)}
𝒗𝒋, where 𝑄 denotes the set of participants.
𝑗 ∈ 𝑆 𝑄 ∗|𝐺| contains the VFOA
target sequence over the course
participant where 𝐺 = {𝑞0, … , 𝑞 𝑄 , 𝑢𝑏𝑐𝑚𝑓, 𝑥ℎ𝑗𝑢𝑓𝑐𝑝𝑏𝑠𝑒, 𝑞𝑠𝑝𝑘𝑓𝑑𝑢𝑗𝑝𝑜 𝑡𝑑𝑠𝑓𝑓𝑜, 𝑣𝑜𝑙𝑜𝑝𝑥𝑜}.
10
We formulate a meeting transcript as a list of triples 𝑌={(𝑞𝑗; 𝑔
𝑗; 𝑣𝑗)}
utterance 𝒗𝒋, where 𝑄 denotes the set of participants. One hot vector.
𝑗 ∈ 𝑆 𝑄 ∗|𝐺| contains the VFOA
target sequence over the course
participant where 𝐺 = {𝑞0, … , 𝑞 𝑄 , 𝑢𝑏𝑐𝑚𝑓, 𝑥ℎ𝑗𝑢𝑓𝑐𝑝𝑏𝑠𝑒, 𝑞𝑠𝑝𝑘𝑓𝑑𝑢𝑗𝑝𝑜 𝑡𝑑𝑠𝑓𝑓𝑜, 𝑣𝑜𝑙𝑜𝑝𝑥𝑜}.
𝑔
𝑗 ∈ 𝑆 𝑄 ∗ 𝐺 , 𝑄 = 4, 𝐺 = 8
11
We formulate a meeting transcript as a list of triples 𝑌={(𝑞𝑗; 𝑔
𝑗; 𝑣𝑗)}
utterance 𝒗𝒋, where 𝑄 denotes the set of participants. One hot vector.
𝑗 ∈ 𝑆 𝑄 ∗|𝐺| contains the VFOA
target sequence over the course
participant where 𝐺 = {𝑞0, … , 𝑞 𝑄 , 𝑢𝑏𝑐𝑚𝑓, 𝑥ℎ𝑗𝑢𝑓𝑐𝑝𝑏𝑠𝑒, 𝑞𝑠𝑝𝑘𝑓𝑑𝑢𝑗𝑝𝑜 𝑡𝑑𝑠𝑓𝑓𝑜, 𝑣𝑜𝑙𝑜𝑝𝑥𝑜}.
12
We formulate a meeting transcript as a list of triples 𝑌={(𝑞𝑗; 𝑔
𝑗; 𝑣𝑗)}
utterance 𝒗𝒋, where 𝑄 denotes the set of participants. One hot vector.
𝑗 ∈ 𝑆 𝑄 ∗|𝐺| contains the VFOA
target sequence over the course
participant where 𝐺 = {𝑞0, … , 𝑞 𝑄 , 𝑢𝑏𝑐𝑚𝑓, 𝑥ℎ𝑗𝑢𝑓𝑐𝑝𝑏𝑠𝑒, 𝑞𝑠𝑝𝑘𝑓𝑑𝑢𝑗𝑝𝑜 𝑡𝑑𝑠𝑓𝑓𝑜, 𝑣𝑜𝑙𝑜𝑝𝑥𝑜}.
13
Hierarchical Attention in Summary Decoder The probability of generating 𝑧𝑗 follows the decoder in PGNet, and 𝛽𝑗𝑘
𝑡𝑣𝑛 is the copying probability.
14
Hierarchical Attention in Summary Decoder The probability of generating 𝑧𝑗 follows the decoder in PGNet, and 𝛽𝑗𝑘
𝑡𝑣𝑛 is the copying probability.
15
16
testing.
17
Dialogue Summarization Video & Audio Audio Only Text Only
Encoder [3]
DiDi Customer Service [2]
18
integrity of dialogue summaries.
net, to generate both key point sequences and the summaries.
19
sentences.
20
Summary Designed rules by domain experts Key point sequence … Totally 51 key points
21
22
23
The relative position of 𝑗-th utterance: where 𝑁 is the utterance number of a dialogue and 𝐿 is the maximum relative position number. 𝐿 is set to 30 in the experiments.
24
The hidden state of the key point decoder is used to guide the generation of the sub-summary instead of the predicted key point. The reason is to avoid the exactly the same initial states for two sub- summaries under the same key point in one key point sequence. e.g., [· · · , Solution, User disapproval, Solution, User approval, · · · ]
25
(use ROUGE-L as reward)
26
27
28
features to enhance the performance.
point sequence to improve the logicality and integrity of the generated summary.
29
30
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.
25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.
arXiv:1811.00185 (2018).
Summarization." arXiv preprint arXiv:1911.12237 (2019).
Conversations." arXiv preprint arXiv:1910.01335 (2019).
Modeling Optimized by Dialogue Acts." 2018 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2018.
machine learning for multimodal interaction. Springer, Berlin, Heidelberg, 2005.
Learning." The World Wide Web Conference. ACM, 2019.
conversations on symptom monitoring”. NAACL 2019.
Conversations.” arXiv preprint arXiv:1910.01335 (2019).
31