Statistical Natural Language Processing An overview of NLP - - PowerPoint PPT Presentation

▶
statistical natural language processing
SMART_READER_LITE
LIVE PREVIEW

Statistical Natural Language Processing An overview of NLP - - PowerPoint PPT Presentation

Sep 13, 2023 793 likes •1.04k views

Statistical Natural Language Processing An overview of NLP applications: some topics not covered during the course ar ltekin University of Tbingen Seminar fr Sprachwissenschaft Summer Semester 2019 Some remarks on the exam

slide-1
SLIDE 1

Statistical Natural Language Processing

An overview of NLP applications: some topics not covered during the course Çağrı Çöltekin

University of Tübingen Seminar für Sprachwissenschaft

Summer Semester 2019

slide-2
SLIDE 2

Some remarks on the exam

fjrst things fjrst

  • Exam is scheduled on Fri July 26, start at 10:00, 10:30, or 11:00?
  • The duration is 2 hours
  • The exam (type of questions, length) will be similar to last year’s exam
  • Topics may shift, covering anything we studied during the course
  • You can bring a ‘cheat sheet’:

– Single a4 paper with anything that you want to remember – You can use both sides – You can hand-write/print as small as you like, but should be legible with bare eye

Questions?

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 1 / 20

slide-3
SLIDE 3

Resit

nobody will need it, but just in case...

  • Note that your fjnal score is combination of

– Exam (40 %) – Assignments, best 6 scores out of 7 (60 %) – Attendance (+ 5 %) – Easter-egg bonus

  • The exam scores will be announced (latest) the week after the exam
  • Last two assignments will be graded in August
  • You can take a resit exam if your overall score <60 %, but you can reach 60 %

by improving your exam score

  • Resit will be scheduled before the beginning of the winter semester. Likely

fjrst (maybe second) week of October

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 2 / 20

slide-4
SLIDE 4

A quick summary so far

Part I Background & machine learning

– Math: linear algebra, probability & information theory – Supervised methods: regression / classifjcation – How evaluate machine learning methods – Neural networks – Sequence learning – Unsupervised learning

Part II NLP methods

– Tokenization / segmentation – N-gram language models – Statistical parsing – Vector representations / vector semantics

Part III (would be) NLP applications

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 3 / 20

slide-5
SLIDE 5

Machine translation

what & why

  • Motivation for MT does not need many words: it is the example you give to

your grandmother when she asks ‘what does a computational linguist do?’

  • Rule-based machine translation is diffjcult
  • Most modern MT systems are statistical

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 4 / 20

slide-6
SLIDE 6

Machine translation

how: basic idea

arg max

e

p(e|f) = arg max

e

p(f|e)p(e)

  • The above defjnes a noisy-channel model
  • p(f|e) estimated with the noisy channel idea
  • p(e) is a language model

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 5 / 20

slide-7
SLIDE 7

Machine translation

how: phrase-based MT

arg max

e

p(e|f) = arg max

e

p(f|e)p(e) Using a parallel corpus,

  • Align sentences, estimate p(f|e)
  • We can estimate p(e) even from a (larger) mono-lingual corpus

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 6 / 20

slide-8
SLIDE 8

Machine translation

how: end-to-end systems (mostly neural)

arg max

e

p(e|f) = arg max

e

p(f|e)p(e) Estimate p(e|f) directly, typically with a recurrent neural network f1 f2 f3 </s> e1 e2 e3 e4 </s> e1 e2 e3 e4 Encoder Decoder

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 7 / 20

slide-9
SLIDE 9

Machine translation

how: end-to-end systems (mostly neural)

arg max

e

p(e|f) = arg max

e

p(f|e)p(e) Estimate p(e|f) directly, typically with a recurrent neural network f1 f2 f3 </s> e1 e2 e3 e4 </s> e1 e2 e3 e4 Encoder Decoder

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 7 / 20

slide-10
SLIDE 10

Machine translation

How does it work? (1)

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 8 / 20

slide-11
SLIDE 11

Machine translation

How does it work? (2)

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 9 / 20

slide-12
SLIDE 12

Machine translation

How does it work? (seriously)

  • Works fjne if you have lots of parallel text
  • A lot of work remains in:

– Solving issues with ambiguities, idioms, special/rare constructions – Low resource languages

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 10 / 20

slide-13
SLIDE 13

Entity recognition

what & why

UN ORG Secretary-General NONE Antonio PER Guterres PER plans NONE to NONE visit NONE Ukraine GEO

  • Many other applications depend on locating certain entities in text
  • Typical entities interest include: people, organizations, locations
  • Can be application specifjc too: e.g., drug/disease names

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 11 / 20

slide-14
SLIDE 14

Entity recognition

how

  • Generally viewed as a typical sequence learning task
  • Any sequence learning model applies: e.g., HMMs, RNNs
  • Some linguistic processing is often helpful (e.g., POS tagging)

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 12 / 20

slide-15
SLIDE 15

Relation extraction

what & why

UN ORG Secretary-General NONE Antonio PER Guterres PER head-of plans NONE to NONE visit NONE Ukraine GEO

  • For many other tasks, we do not only need entities, but the relations between

them

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 13 / 20

slide-16
SLIDE 16

Relation extraction

how

  • Many approaches rely on patterns
  • Using classifjers on annotated data is also popular
  • 1. Extract all pairs of entities of interest
  • 2. Train the classifjer, to predict whether the entities are related
  • Semi-supervised learning methods are common
  • Does it also look like dependency parsing?

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 14 / 20

slide-17
SLIDE 17

Summarization

what & why

  • We have lots, lots of text on any subject of choice
  • Probably you use them daily (e.g., news aggregators), but applications of

summarization are much wider

  • Summarization

– reduces the reading time – helps selecting right documents to read – may improve/help with

  • indexing
  • storing/processing/searching large document collections
  • other applications like question answering

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 15 / 20

slide-18
SLIDE 18

Summarization

how

Extractive summarization selects important sentences from the text.

  • The task is binary classifjcation (paying attention to the

sequence)

  • Classifjer decides whether to keep or discard the sentence in the

summary Abstractive summarization fuses sentences, combining and re-structuring them How about treating it like a machine translation problem?

  • RNNs of the sort used in MT have lately been popular for

summarization too

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 16 / 20

slide-19
SLIDE 19

Question answering

what & why

  • QA is another NLP application that needs little explanation
  • The task is given a question fjnd the answer in a database, or a unstructured

document collection

  • Domain specifjc specifjc are common
  • More general QA systems can perform well, sometimes better than humans

(e.g., IBM Watson)

  • Also an important part of for modern personal assistant systems
  • Most systems are complex, combining many of the methods we discussed in

the class (and more)

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 17 / 20

slide-20
SLIDE 20

Question answering

how

  • The natural language questions are turned int formal queries, searched in a

database

– linguistic processing (parsing) helps – Supervised methods can learn queries from natural language questions

  • Again, RNNs have been recent popular approach

Question Text with answer RNN RNN Answer

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 18 / 20

slide-21
SLIDE 21

More…

  • Topic modeling / text mining
  • Information extraction
  • Coreference resolution
  • Semantic role labeling
  • Dialog systems
  • Speech recognition
  • Speech synthesis
  • Spelling correction
  • Text normalization

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 19 / 20

slide-22
SLIDE 22

Summary

  • Many other problems/applications in NLP can be solved with the methods

we studied in this course

  • Most of the real-world problems require a combination of multiple methods

Next: Mon Summary & your questions Wed Assignments 6 & 7, exam questions/discussion Fri Exam

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 20 / 20

slide-23
SLIDE 23

Summary

  • Many other problems/applications in NLP can be solved with the methods

we studied in this course

  • Most of the real-world problems require a combination of multiple methods

Next: Mon Summary & your questions Wed Assignments 6 & 7, exam questions/discussion Fri Exam

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 20 / 20

slide-24
SLIDE 24

Additional reading, references, credits

  • The textbook (Jurafsky and Martin 2009) includes detailed information on

many of these problems/applications (more on the 3rd edition draft)

Ç. Çöltekin, SfS / University of Tübingen Summer Semester 2019 A.1