Adversarial network for natural language systhesis Speaker - PowerPoint PPT Presentation

Adversarial network for natural language systhesis

Speaker introduction Rajib Biswas Area of interest • Lead Data Scientist in Ericsson, Bangalore. • NLP - Virtual Assistant, Knowledge Graphs • Working with a global AI team for AI • Computer Vision - Image classification. driven network design and optimization. • Time series and forecasting. • Have worked with Intel, Samsung, • Recommendation system. Fidelity Investments earlier. • Have filed few patents related to AI. � 2

Agenda 1. Language generation : now and future. 2. Introduction to GAN. 3. Challenges with generative model for text 4. GAN models for text. 5. Task specific Language Generation 6. Evaluation � 3

Language generation: now & future � 4

Language generation: now & future Language Generation Language understanding Language processing � 5

Language generation: now & future Applications: • Auto content curation for editors. � 6

Language generation: now & future Applications: • Auto report generation from BI apps � 7

Language generation: now & future Applications: • Virtual Assistant /chatbots � 8

Introduction to GAN Real image of Ian Goodfellow Synthetic fake images by GAN . https://thispersondoesnotexist.com/ � 9

Introduction to GAN GAN(Generative Adversarial Network) - generator network( G ) trained to produce realistic samples by introducing “adversary" [a discriminator network( D ) ] � 10

Introduction to GAN GAN(Generative Adversarial Network) - generator network( G ) trained to produce realistic samples by introducing “adversary" [a discriminator network( D ) ] - D detects if a given sample is ‘real’ or ‘fake’. - D dynamically update evaluation metric for tuning the generator. - until D will output with probability 0.5 for both classes ; obtain ‘Nash Equilibrium’. � 11

Introduction to GAN Objective: 
 Distinguish between real and fake Objective: 
 Generate image close to real � 12

Introduction to GAN Min-max game played by two network, whose Value function is given Output for real image Output for fake image Correct identification Getting caught by D D(x) = Probability that, x is real according to D. G(z) = sample generated by G, given latent vector Z. � 13

Challenges with GAN for text GAN is not natural fit for discrete data such as text. Training of generator is difficult due to discreteness of text data. � 14

Challenges with GAN for text GAN is not natural fit for discrete data such as text. Training of generator is difficult due to discreteness of text data. Non-Differentiability at output of generator. Non-differentiable for discrete data Generator Discriminator argmax : 
 min: 1- D(G(z)) P (softmax(h)) X Back-propagation � 15

GAN models for text Gumbel-softmax trick h t = hidden state of RNN generator at step t g t = sample from Gumbel distribution. 𝜐 = parameter to control how close the continuous approx distribution to the discrete one. 𝜐 → 0, y is close approximation to one-hot vector. y t = softmax ( (h+g)/ 𝜐 ) Kusner et al: GANs for sequence of discrete elements with Gumbel-Softmax Distribution � 16

GAN models for text Problem so far: Sampling of tokens from discrete spaces. Solution proposed: Sample single token ‘sentence vector’ from discrete continuous space of all sentence vectors. � 17

GAN models for text: AutoEncoder Problem so far: Sampling of tokens from discrete spaces. Solution proposed: Sample single token ‘sentence vector’ from discrete continuous space of all sentence vectors. Get sentence vector of real sentences by training a Auto-Encoder . Latent space GAN David et al. “Adversarial Text Generation Without Reinforcement Learning”. � 18

GAN models for text Let’s play a game. Today , the sky is ______ in London.I _____ this weather. What are the possible words coming to your mind? � 19

GAN models for text Let’s play a game. Today , the sky is ______ in London.I _____ this weather. cloudy 
 love 
 sunny 
 hate clear What are the possible words coming to your mind? � 20

GAN models for text : SeqGAN Text Generation is a sequential decision-making process : RL Agent : Generator State : text generated so far Action : predicting next word, based on context (previous state). Reward : Prediction score(real/fake) from Discriminator Policy ( 𝜌 ): Which action to pick at any state? policy function: 𝜌 (a | s, 𝜄 ) Objective : Find optimal policy 𝜌 * => optimal 𝜄 SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient.(Lantao You et al) � 21

GAN models for text : SeqGAN SeqGAN • D : input: sentence • output :reward score of realness of this sentence. • Provided back to G to update policy at end of episode. • G :Input:sequence of words, • output: probability distribution over next word. • h t = RNN(h t-1 , x t ) • p(a t | x 1 ,…,x t ) = z( h t )= softmax( b + W*h t ) � 22

GAN models for text : SeqGAN Objective : Find optimal policy 𝜌 * => optimal 𝜄 Policy gradient REINFORCE update 𝛼𝜌 (A t | S t , 𝜄 t ) = Vector indicates direction of max increase of prob of action A t , when state S t is encountered. G t : Cumulative reward , while following Policy 𝜌 . Indicates amount of movement. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient.(Lantao You et al) � 23

GAN models for text : SeqGAN How to determine reward before sentences are completed? -SeqGAN applies Monte-Carlo search to roll-out current policy to estimate the reward. - Generator uses current learned policy network to roll-out iteratively till end of sentences to get estimated reward. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient.(Lantao You et al) � 24

GAN models for text : LeakGAN LeakGAN • Long sentence generation is hard, as intermediate reward estimation is noisy. • LeakGan solves that by introducing Worker(Criminal) and Manager(Spy). • Worker : creates fake sample • Discriminator(Police): detects fake/real • Manager: act as spy and leaks features used to identify fake sample by Discriminator. � 25

GAN models for text Leveraging more information from discriminator , to generate Better quality text , may cause another critical issue !! Mode Collapse: exhibit poor diversity amongst generated samples UnrollGAN, Luke te.al. ICRL 2017 � 26

GAN models for text : MaskGAN Leveraging more information from discriminator to generate Better quality text , may cause another critical issue !! Mode Collapse: exhibit poor diversity amongst generated samples MaskGAN claims to reduce mode collapse and help with training stability. This method shows evidence that it produces more realistic text samples compared to a maximum likelihood trained model. � 27

GAN models for text : MaskGAN Encoder encodes masked text input, 
 Discriminator also uses 
 Generator learns to fill in the blank by reward score same architecture Provided by the Discriminator. MaskGAN : Better Text Generation via Filling in the______. (William Fedus et al, ICRL 2018) � 28

GAN models for text Challenges with RL based methods 1.Unstable training process due High variance . Because few samples to estimate gradient of policy. 2.Policy gradient methods tend to converge to local maxima, when state- action space is huge. Action choices= |V| , vocabulary size. � 29

Evaluation How to measure performance of metrics? GANs are not optimised for traditional cross-entropy loss (unlike MLE), so usually ‘loss’ is not used as performance metrics. BLEU(Bilingual Evaluation Understudy Score): Counting matching n-grams between generated and target sentence. BLEU doesn’t cover all aspect of language correctness. � 30

Evaluation LeakGAN performs well with BLEU score. MaskGAN performs well with self-BLEU score(detects mode collapse.) Left : BLEU score on EMNLP 2017 WMT, Right : self-BLEU score � 31

Evaluation Benchmark on standard datasets Human preference score of paired comparison on IMDB and PTB datasets � 32

Evaluation: results Dataset Epoch Output from SeqGAN PTB 5 Employees may cure they were considering the agency that’s two congress cases ms. Jhonson clearly noted that began growth PTB 10 Can end of its criminal office charges to remove the pacific law which is all the <unk> response to PTB 20 Capital offers flat the debt carrier to imports from <unk> mr. George said it expects net sales to reduce � 33

Evaluation: results Conditional samples from PTB for both MaskGAN and MaskMLE � 34

Task specific Language generation Dialogue Generation Input That’s our wake up call Vanilla-MLE We ‘re gonna be late for the meeting Reinforce I ‘ll be right back REGS MC We ‘re gonna have to get to the station Adversarial Learning for Neural Dialogue Generation, Jiwei et al � 35