SLIDE 1
Non-Monotonic Sequential Text Generation
Sean Welleck, Kianté Brantley, Hal Daumé III, Kyunghyun Cho
Sequential Text Generation
Y = (y1, y2, …… , yN)
(hi, how, are, you, ?)
Non-Monotonic Sequential Text Generation Sean Welleck, Kiant - - PowerPoint PPT Presentation
Non-Monotonic Sequential Text Generation Sean Welleck, Kiant - - PowerPoint PPT Presentation
Non-Monotonic Sequential Text Generation Sean Welleck, Kiant Brantley, Hal Daum III, Kyunghyun Cho Sequential Text Generation Y = ( y 1 , y 2 , , y N ) ( hi , how , are , you , ? ) Sequential Text Generation Unconditional Y ( hi ,
SLIDE 2
SLIDE 3
Sequential Text Generation Unconditional
∼
(hi, how, are, you, ?) (good, to, see, you, !) (what, time, is, it, ?)
…
Y
Policy
π
SLIDE 4
Sequential Text Generation Conditional
→ (how, are, you, ?)
Y
元気ですか?
X
→ Policy
π
Transformer, LSTM, …
SLIDE 5
Sequential Text Generation Monotonic
how
π(a1|s1)
are
π(a2|s2) π(a3|s3)
you ?
π(a4|s4)
token (how, are, X)
SLIDE 6
Sequential Text Generation Non-Monotonic
how
π(a1|s1)
are you ?
π(a2|s2) π(a3|s3) π(a4|s4)
are how ? you how are you ?
SLIDE 7
…, how, are, you , ?, the, …
Binary Tree Generating Policy
[ ] are
…., you , ?, … …., how , …
[ ]
SLIDE 8
∅ ∅ ∅ ∅
…., you , …
Binary Tree Generating Policy
how you ? ∅
…, how, are, you , ?, the, …
are
…., how , … …., you , ?, …
SLIDE 9
Binary Tree Generating Policy how are you ? are how ? you
∅ ∅ ∅ ∅ ∅
in-order traversal
∅ ∅ ∅ ∅ how you ∅ ? are
SLIDE 10
Binary Tree Generating Policy
how are you ? ∅ ∅ ∅ ∅ ∅ how are you ? ∅ ∅ ∅ ∅ ∅ how are you ? ∅ ∅ ∅ ∅ ∅
… …
SLIDE 11
Imitation Learning
Define an oracle Sample sequences Minimize cost
π*(at|st, X, Y) (a1, …, aT) ∼ π* KL [π*( ⋅ |st), πθ( ⋅ |st)]
SLIDE 12
Oracles
Oracle: only puts mass on valid actions
A B C D ∅ ∅ ∅ ∅ ∅
A B C D E A B C D E A B C D E A B C D E
π*
uniform
SLIDE 13
Oracles
Oracle: only puts mass on valid actions
A B C D ∅ ∅ ∅ ∅ ∅
A B C D E A B C D E A B C D E A B C D E
π*
uniform
ℒ1 = KL( , )
A B C D E
π* πθ
uniform
A B C D E
SLIDE 14
Oracles
left-right: only put mass on ‘left-most’ valid action
A B C D ∅ ∅ ∅
A B C D E A B C D E A B C D E A B C D E
∅ ∅
π*
left-right
SLIDE 15
Coaching
Weight correct actions by the learned policy
A C ∅
A B C D E A B C D E
⊙
A B C D E
∝ … …
π*
uniform
πθ π*
coaching
SLIDE 16
Coaching
Weight valid actions by the learned policy Loss reinforces preferred orders
A C ∅
A B C D E A B C D E
⊙
A B C D E
∝ … …
π*
uniform
πθ π*
coaching
KL( , )
A B C D E A B C D E
π*
coaching
πθ
SLIDE 17
Results | Unconditional
SLIDE 18
Results | Unconditional
SLIDE 19
Results | Conditional
Word Reordering
SLIDE 20
Results | Conditional
Machine Translation
SLIDE 21
π( ⋅ | ) ∼
Left-Right
π( ⋅ | ) ∼
Non-Monotonic
…
Results | Variable-Sized Text Infilling
SLIDE 22
Results | Variable-Sized Text Infilling
SLIDE 23
- Code & Pre-trained Models:
https://github.com/wellecks/nonmonotonic_text
- Poster #45 (Pacific Ballroom)
SLIDE 24
- Code & Pre-trained Models:
https://github.com/wellecks/nonmonotonic_text
- Poster #45 (Pacific Ballroom)