Algorithms for NLP IITP, Spring 2020 HMMs, POS tagging, NER Yulia - PowerPoint PPT Presentation

Algorithms for NLP IITP, Spring 2020 HMMs, POS tagging, NER Yulia Tsvetkov 1

Plan ▪ POS tagging recap ▪ HMMs, Viterbi ▪ HMMs+ ▪ dealing with UNKs ▪ 3gram HMMs ▪ multilingual POS tagging ▪ Featurizing HMMs ▪ MEMM, CRF ▪ NER ▪ HMMs is speech recognition 2

https://universaldependencies.org

●

▪ ▪ ▪ ▪ → → ▪ ▪ ▪

Levels of linguistic knowledge Slide credit: Noah Smith 15

Sequence Labeling ▪ map a sequence of words to a sequence of labels ▪ Part-of-speech tagging (Church, 1988; Brants, 2000) ▪ Named entity recognition (Bikel et al., 1999) ▪ Text chunking and shallow parsing (Ramshaw and Marcus, 1995) ▪ Word alignment of parallel text (Vogel et al., 1996) ▪ Compression (Conroy and O’Leary, 2001) ▪ Acoustic models, discourse segmentation, etc. 16

Sequence labeling as classification 17

the future is independent of the past given the present

the future is independent of the past given the present

▪ ▪ ... ▪ o 1 o 2 o n

▪

▪ ▪ ▪ ▪ ▪ ▪

▪ ▪ ▪

▪ ▪ → ▪ → ▪ → ▪ → ▪ → ▪ → ▪ ▪ → ▪ ▪ ▪

▪ ▪ ▪ ▪

▪ ▪

▪ ▪

▪

▪ ▪ ▪

▪ ▪ ▪ ▪ ▪

▪ ▪

▪ ▪ ▪ ▪ ⇒ ▪

▪ ▪ ▪

▪ ▪ ▪

▪ ▪ ▪ ▪

“speech lab” ssssssssppppeeeeeeetshshshshllllaeaeaebbbbb