Sequence Labeling, Contd Prof. Sameer Singh CS 295: STATISTICAL NLP - - PowerPoint PPT Presentation

▶
sequence labeling contd
SMART_READER_LITE
LIVE PREVIEW

Sequence Labeling, Contd Prof. Sameer Singh CS 295: STATISTICAL NLP - - PowerPoint PPT Presentation

Feb 21, 2023 111 likes •418 views

Sequence Labeling, Contd Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February 2, 2017 Based on slides from Dan Klein, and everyone else he copied from. Outline Marginal Inference in HMMs: F/B algorithm Maximum Entropy Markov

slide-1
SLIDE 1

Sequence Labeling, Contd

  • Prof. Sameer Singh

CS 295: STATISTICAL NLP WINTER 2017

February 2, 2017

Based on slides from Dan Klein, and everyone else he copied from.

slide-2
SLIDE 2

Outline

CS 295: STATISTICAL NLP (WINTER 2017) 2

Marginal Inference in HMMs: F/B algorithm Maximum Entropy Markov Models Conditional Random Fields Neural Sequence Tagging

slide-3
SLIDE 3

Outline

CS 295: STATISTICAL NLP (WINTER 2017) 3

Marginal Inference in HMMs: F/B algorithm Maximum Entropy Markov Models Conditional Random Fields Neural Sequence Tagging

slide-4
SLIDE 4

Expectation Maximization

CS 295: STATISTICAL NLP (WINTER 2017) 4

K-Means Initialization Pick K random centroids Label Data from the Model Cluster all the points Update the Model from Data Update centroids

slide-5
SLIDE 5

Label Data from the Model

CS 295: STATISTICAL NLP (WINTER 2017) 5

Hard-EM Soft-EM

slide-6
SLIDE 6

Dynamic Programming

CS 295: STATISTICAL NLP (WINTER 2017) 6

slide-7
SLIDE 7

Forward Backward Algorithm

CS 295: STATISTICAL NLP (WINTER 2017) 7

slide-8
SLIDE 8

Updating the Model from Data

CS 295: STATISTICAL NLP (WINTER 2017) 8

Hard-EM Soft-EM

slide-9
SLIDE 9

Outline

CS 295: STATISTICAL NLP (WINTER 2017) 9

Marginal Inference in HMMs: F/B algorithm Maximum Entropy Markov Models Conditional Random Fields Neural Sequence Tagging

slide-10
SLIDE 10

Named Entity Recognition

CS 295: STATISTICAL NLP (WINTER 2017) 10

George W. Bush spoke from the White House today . PER PER PER O O O LOC LOC O O B-PER I-PER I-PER O O O B-LOC I-LOC O O B-PER I-PER E-PER O O O B-LOC E-LOC O O

slide-11
SLIDE 11
  • Max. Entropy Markov Models

CS 295: STATISTICAL NLP (WINTER 2017) 11

slide-12
SLIDE 12

Graphical Model Notation

CS 295: STATISTICAL NLP (WINTER 2017) 12

S E HMMs S E MEMMs

slide-13
SLIDE 13

Adding Features (for POS)

CS 295: STATISTICAL NLP (WINTER 2017) 13

Current Word Window Words

slide-14
SLIDE 14

Adding Features

CS 295: STATISTICAL NLP (WINTER 2017) 14

S E

slide-15
SLIDE 15

Predictions Using MEMMs

CS 295: STATISTICAL NLP (WINTER 2017) 15

Greedy Viterbi Decoding

slide-16
SLIDE 16

Training MEMMs

CS 295: STATISTICAL NLP (WINTER 2017) 16

slide-17
SLIDE 17

Outline

CS 295: STATISTICAL NLP (WINTER 2017) 17

Marginal Inference in HMMs: F/B algorithm Maximum Entropy Markov Models Conditional Random Fields Neural Sequence Tagging

slide-18
SLIDE 18

Label Bias Problem

CS 295: STATISTICAL NLP (WINTER 2017) 18

S O O O I I I B B B E B B

slide-19
SLIDE 19

Conditional Random Fields

CS 295: STATISTICAL NLP (WINTER 2017) 19

slide-20
SLIDE 20

Graphical Model Notation

CS 295: STATISTICAL NLP (WINTER 2017) 20

S E HMMs/MEMMs S E CRFs

slide-21
SLIDE 21

Predictions Using CRFs

CS 295: STATISTICAL NLP (WINTER 2017) 21

slide-22
SLIDE 22

Likelihood Training of CRFs

CS 295: STATISTICAL NLP (WINTER 2017) 22

slide-23
SLIDE 23

Likelihood Training of CRFs

CS 295: STATISTICAL NLP (WINTER 2017) 23

slide-24
SLIDE 24

Forward-Backward Algorithm

CS 295: STATISTICAL NLP (WINTER 2017) 24

slide-25
SLIDE 25

Outline

CS 295: STATISTICAL NLP (WINTER 2017) 25

Marginal Inference in HMMs: F/B algorithm Maximum Entropy Markov Models Conditional Random Fields Neural Sequence Tagging

slide-26
SLIDE 26

Simple Neural Tagger

CS 295: STATISTICAL NLP (WINTER 2017) 26

PRP I VB love NN food Start End

slide-27
SLIDE 27

MEMM-ish Neural Tagger

CS 295: STATISTICAL NLP (WINTER 2017) 27

PRP I VB love NN food Start End S E

slide-28
SLIDE 28

Recurrent Neural Tagger

CS 295: STATISTICAL NLP (WINTER 2017) 28

PRP I VB love NN food

slide-29
SLIDE 29

Bidirectional RNN Tagger

CS 295: STATISTICAL NLP (WINTER 2017) 29

PRP I VB love NN food

slide-30
SLIDE 30

Upcoming…

CS 295: STATISTICAL NLP (WINTER 2017) 30

  • Homework 2 is due (~10 days): February 13, 2017
  • Write-up, data, and code for Homework 2 is up
  • Ask questions early!

Homework

  • Proposal is due on Tuesday: February 7, 2017
  • Only 2 pages

Project

  • Paper summaries: February 17, February 28, March 14
  • Only 1 page each

Summaries