Natural Language Processing (NLP) In 11-711 Algorithms for NLP we - - PowerPoint PPT Presentation

natural language processing nlp
SMART_READER_LITE
LIVE PREVIEW

Natural Language Processing (NLP) In 11-711 Algorithms for NLP we - - PowerPoint PPT Presentation

Natural Language Processing (NLP) In 11-711 Algorithms for NLP we take an English-centric approach to NLP This enables us to work with a language that all of us understand and focus on core algorithms and tasks Even


slide-1
SLIDE 1

Natural Language Processing (NLP)

  • In 11-711 “Algorithms for NLP” we take an

English-centric approach to NLP

○ This enables us to work with a language that all of us understand and focus on core algorithms and tasks

  • Even English-centric NLP is difficult!
slide-2
SLIDE 2

English Natural Language Processing (NLP)

A conversational agent contains

  • Speech recognition
  • Language analysis

○ Language modelling, spelling correction ○ Syntactic analysis: part-of-speech tagging, syntactic parsing ○ Semantic analysis: named-entity recognition, event detection, word sense disambiguation, semantic role labelling ○ Longer range semantic analysis: coreference resolution, entity linking ○ etc.

  • Dialog processing

○ Discourse analysis, user adaptation, etc.

  • Information retrieval
  • Text to speech
slide-3
SLIDE 3

But most of the world today is multilingual

Source: Ethnologue Source: US Census Bureau

slide-4
SLIDE 4

World’s Englishes

slide-5
SLIDE 5

NLP beyond English

  • ~7,000 languages
  • thousands of language

varieties

slide-6
SLIDE 6

Tokenization

slide-7
SLIDE 7

Part-of-speech tagging

slide-8
SLIDE 8

Tokenization + disambiguation

slide-9
SLIDE 9

Tokenization + disambiguation

slide-10
SLIDE 10

Morphosyntactic analysis

slide-11
SLIDE 11

Morphological processing

slide-12
SLIDE 12

Syntactic parsing

slide-13
SLIDE 13
  • Every language “sees” the world in a different way
  • For example, it could depend on cultural or historical conditions
  • Russian has very few words for colors, Japanese has hundreds
  • Multiword expressions, e.g. it’s raining cats and dogs or wake up and

metaphors, e.g. love is a journey are very different across languages

Semantic analysis

slide-14
SLIDE 14
slide-15
SLIDE 15

Multilingual NLP

  • Levels of linguistic structure
  • Categorization of languages and processing of linguistic structures across

languages

  • Multilingual modeling