Syntactic Processing: Parts-of-Speech Tagging CSE354 - Spring 2020 - - PowerPoint PPT Presentation

▶
syntactic processing parts of speech tagging
SMART_READER_LITE
LIVE PREVIEW

Syntactic Processing: Parts-of-Speech Tagging CSE354 - Spring 2020 - - PowerPoint PPT Presentation

Aug 02, 2023 658 likes •959 views

Syntactic Processing: Parts-of-Speech Tagging CSE354 - Spring 2020 Task Syntactic Processing Machine learning: h o w ? Parts-of-Speech Tagging Logistic regression Parts-of-Speech Open Class: Nouns, Verbs, Adjectives,

slide-1
SLIDE 1

Syntactic Processing: Parts-of-Speech Tagging

CSE354 - Spring 2020

slide-2
SLIDE 2

h

  • w

? Task

  • Syntactic Processing

Parts-of-Speech Tagging

  • Machine learning:

○ Logistic regression

slide-3
SLIDE 3

Parts-of-Speech

Open Class: Nouns, Verbs, Adjectives, Adverbs

slide-4
SLIDE 4

Parts-of-Speech

Open Class: Nouns, Verbs, Adjectives, Adverbs Function words: Determiners, conjunctions, pronouns, prepositions

slide-5
SLIDE 5

Parts-of-Speech: The Penn Treebank Tagset

slide-6
SLIDE 6

Parts-of-Speech: Social Media Tagset

(Gimpel et al., 2010)

slide-7
SLIDE 7

POS Tagging: Applications

  • Resolving ambiguity (speech: “lead”)
  • Shallow searching: find noun phrases
  • Speed up parsing
  • Use as feature (or in place of word)
slide-8
SLIDE 8

POS Tagging: Applications

  • Resolving ambiguity (speech: “lead”)
  • Shallow searching: find noun phrases
  • Speed up parsing
  • Use as feature (or in place of word)

For this course:

  • An introduction to language-based classification (logistic regression)
  • Understand what modern deep learning methods are dealing with implicitly.
slide-9
SLIDE 9

Window-based POS Tagging

?

The book looks brief so I am happy .

slide-10
SLIDE 10

Window-based POS Tagging

D

The book looks brief so I am happy .

slide-11
SLIDE 11

Window-based POS Tagging

N D

The book looks brief so I am happy .

slide-12
SLIDE 12

Window-based POS Tagging

? N D

The book looks brief so I am happy .

slide-13
SLIDE 13

Window-based POS Tagging

V N D

The book looks brief so I am happy .

slide-14
SLIDE 14

Window-based POS Tagging

A N D V

The book looks brief so I am happy .

slide-15
SLIDE 15

Window-based POS Tagging

? N D V

The book looks brief so I am happy .

slide-16
SLIDE 16

Window-based POS Tagging

window size

  • f 3

? N D V

The book looks brief so I am happy .

slide-17
SLIDE 17

Window-based POS Tagging

window size

  • f 3

? N D V

The book looks brief so I am happy .

slide-18
SLIDE 18

window size

  • f 3

? N D V

P(posi = ‘N’|wordi = “brief”) = 0.3

The book looks brief so I am happy .

Window-based POS Tagging

slide-19
SLIDE 19

window size

  • f 3

? N D V

P(posi = ‘N’|wordi = “brief”) = 0.3 P(posi = ‘V’|wordi = “brief”) = 0.4 P(posi = ‘A’|wordi = “brief”) = 0.3

The book looks brief so I am happy .

Window-based POS Tagging

slide-20
SLIDE 20

window size

  • f 3

? N D V

P(pi=‘N’|wi=brief) = .30 P(pi=‘V’|wi=brief) = .40 P(pi=‘A’|wi=brief) = .30

The book looks brief so I am happy .

Window-based POS Tagging

slide-21
SLIDE 21

window size

  • f 3

? N D V

P(pi=‘N’|wi=brief,wi-1=looks,wi+1=so) = ?? P(pi=‘V’|wi=brief,wi-1=looks,wi+1=so) = ?? P(pi=‘A’|wi=brief,wi-1=looks,wi+1=so) = ??

The book looks brief so I am happy .

Window-based POS Tagging

slide-22
SLIDE 22

window size

  • f 3

? N D V

P(pi=‘N’|wi=brief,wi-1=looks,wi+1=so) = .005 P(pi=‘V’|wi=brief,wi-1=looks,wi+1=so) = .005 P(pi=‘A’|wi=brief,wi-1=looks,wi+1=so) = .99

The book looks brief so I am happy .

Window-based POS Tagging

ideal result

slide-23
SLIDE 23

window size

  • f 3

? N D V

P(pi=‘N’|wi=brief,wi-1=looks,wi+1=so) = .3 P(pi=‘V’|wi=brief,wi-1=looks,wi+1=so) = .4 P(pi=‘A’|wi=brief,wi-1=looks,wi+1=so) = .3

The book looks brief so I am happy .

Window-based POS Tagging

More likely, because we haven’t seen this context before.

slide-24
SLIDE 24

window size

  • f 3

? N D V

P(pi=‘N’|wi=brief,wi-1=looks,wi+1=so) = .3 P(pi=‘V’|wi=brief,wi-1=looks,wi+1=so) = .4 P(pi=‘A’|wi=brief,wi-1=looks,wi+1=so) = .3

The book looks brief so I am happy .

Window-based POS Tagging

More likely, because we haven’t seen this context before.

slide-25
SLIDE 25

Sequential Model

window size

  • f 3

? N D V

sequence

  • rder of 1

The book looks brief so I am happy .

P(pi=‘N’|wi=brief,wi-1=looks,wi+1=so) = .3 P(pi=‘V’|wi=brief,wi-1=looks,wi+1=so) = .4 P(pi=‘A’|wi=brief,wi-1=looks,wi+1=so) = .3

slide-26
SLIDE 26

Sequential Model

window size

  • f 3

? N D V

sequence

  • rder of 1

The book looks brief so I am happy .

P(pi=‘N’|wi=brief,wi-1=looks,wi+1=so) = .3 P(pi=‘V’|wi=brief,wi-1=looks,wi+1=so) = .4 P(pi=‘A’|wi=brief,wi-1=looks,wi+1=so) = .3

slide-27
SLIDE 27

Sequential Model

window size

  • f 3

? N D V

sequence

  • rder of 1

The book looks brief so I am happy .

P(pi=‘N’|pi-1=V) = .4 P(pi=‘V’|pi-1=V) = .10 P(pi=‘A’|pi-1=V) = .4

slide-28
SLIDE 28

Sequential Model

window size

  • f 3

? N D V

sequence

  • rder of 1

The book looks brief so I am happy .

P(pi=‘N’|pi-1=V,wi=brief) = .3 P(pi=‘V’|pi-1=V,wi=brief) = .05 P(pi=‘A’|pi-1=V,wi=brief) = .65

slide-29
SLIDE 29

Sequence modeling

  • - Tasks that in which a current label is dependent on previous

labels within a sequence. More generally: tasks that can leverage the order of words. Most basic example: Language Modeling

  • - Predicting the next word given previous.