unsupervised language learning representation learning
play

Unsupervised Language Learning: Representation Learning for NLP - PowerPoint PPT Presentation

Mar 02, 2024 •127 likes •898 views

Unsupervised Language Learning: Representation Learning for NLP Unsupervised Language Learning: Representation Learning for NLP Katia Shutova ILLC University of Amsterdam 3 April 2018 Unsupervised Language Learning: Representation Learning

  1. Unsupervised Language Learning: Representation Learning for NLP Unsupervised Language Learning: Representation Learning for NLP Katia Shutova ILLC University of Amsterdam 3 April 2018

  2. Unsupervised Language Learning: Representation Learning for NLP Taught by... ◮ Lecturers: Katia Shutova and Wilker Aziz ◮ Teaching assistant: Samira Abnar

  3. Unsupervised Language Learning: Representation Learning for NLP Lecture 1: Introduction Overview of the course Distributional semantics Count-based models Similarity Distributional word clustering

  4. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Overview of the course ◮ This course is about learning meaning representations ◮ Methods for learning meaning representations from linguistic data ◮ Analysis of meaning representations learnt ◮ Applications ◮ This is a research seminar ◮ Lectures ◮ You will present and critique research papers, ◮ implement and evaluate representation learning methods ◮ and analyse their behaviour

  5. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Overview of the course We will cover the following topics: ◮ Introduction to distributional semantics ◮ Learning word and phrase representations – deep learning ◮ Learning word representations – Bayesian learning ◮ Multilingual word representations ◮ Multimodal word representations (language and vision) ◮ Applications: NLP and neuroscience

  6. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Assessment Work in groups of 2. ◮ Presentation and participation (20%) ◮ Present 1 paper per group in class ◮ Practical assignments, assessed by reports 1. Analysis of the properties of word representations (10%) 2. Implement 3 representation learning methods (20%) 3. Evaluate in the context of external NLP applications – final report (50%) More information at the first lab session on Thursday, 5 April.

  7. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Also note: Course materials and more info: https://uva-slpl.github.io/ull/ Contact ◮ Main contact – Samira: s.abnar@uva.nl ◮ Katia: e.shutova@uva.nl ◮ Wilker: w.aziz@uva.nl Email Samira by Thursday, 5 April with details of your group. ◮ names of the students ◮ their email addresses ◮ subject: ULL group assignment

  8. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Natural Language Processing Many popular applications ◮ Information retrieval ◮ Machine translation ◮ Question answering ◮ Dialogue systems ◮ Sentiment analysis ◮ Recently: fact checking etc.

  9. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Why is NLP difficult? Similar strings mean different things, different strings mean the same thing. ◮ Synonymy: different strings can mean the same thing The King’s speech gave the much needed reassurance to his people. His majesty’s address reassured the crowds. ◮ Ambiguity: same strings can mean different things His majesty’s address reassured the crowds. His majesty’s address is Buckingham Palace, London SW1A 1AA.

  10. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Why is NLP difficult? Similar strings mean different things, different strings mean the same thing. ◮ Synonymy: different strings can mean the same thing The King’s speech gave the much needed reassurance to his people. His majesty’s address reassured the crowds. ◮ Ambiguity: same strings can mean different things His majesty’s address reassured the crowds. His majesty’s address is Buckingham Palace, London SW1A 1AA.

  11. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Why is NLP difficult? Similar strings mean different things, different strings mean the same thing. ◮ Synonymy: different strings can mean the same thing The King’s speech gave the much needed reassurance to his people. His majesty’s address reassured the crowds. ◮ Ambiguity: same strings can mean different things His majesty’s address reassured the crowds. His majesty’s address is Buckingham Palace, London SW1A 1AA.

  12. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Why is NLP difficult? Similar strings mean different things, different strings mean the same thing. ◮ Synonymy: different strings can mean the same thing The King’s speech gave the much needed reassurance to his people. His majesty’s address reassured the crowds. ◮ Ambiguity: same strings can mean different things His majesty’s address reassured the crowds. His majesty’s address is Buckingham Palace, London SW1A 1AA.

  13. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Why is NLP difficult? Similar strings mean different things, different strings mean the same thing. ◮ Synonymy: different strings can mean the same thing The King’s speech gave the much needed reassurance to his people. His majesty’s address reassured the crowds. ◮ Ambiguity: same strings can mean different things His majesty’s address reassured the crowds. His majesty’s address is Buckingham Palace, London SW1A 1AA.

  14. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Wouldn’t it be better if . . . ? The properties which make natural language difficult to process are essential to human communication: ◮ Flexible ◮ Learnable, but expressive and compact ◮ Emergent, evolving systems Synonymy and ambiguity go along with these properties. Natural language communication can be indefinitely precise: ◮ Ambiguity is mostly local (for humans) ◮ resolved by immediate context ◮ but requires world knowledge

  15. Unsupervised Language Learning: Representation Learning for NLP Overview of the course Wouldn’t it be better if . . . ? The properties which make natural language difficult to process are essential to human communication: ◮ Flexible ◮ Learnable, but expressive and compact ◮ Emergent, evolving systems Synonymy and ambiguity go along with these properties. Natural language communication can be indefinitely precise: ◮ Ambiguity is mostly local (for humans) ◮ resolved by immediate context ◮ but requires world knowledge

  16. Unsupervised Language Learning: Representation Learning for NLP Overview of the course World knowledge... ◮ Impossible to hand-code at a large-scale ◮ either limited domain applications ◮ or learn approximations from the data

  17. Unsupervised Language Learning: Representation Learning for NLP Distributional semantics Distributional hypothesis You shall know a word by the company it keeps (Firth) The meaning of a word is defined by the way it is used (Wittgenstein). it was authentic scrumpy, rather sharp and very strong we could taste a famous local product — scrumpy spending hours in the pub drinking scrumpy Cornish Scrumpy Medium Dry. £19.28 - Case

  18. Unsupervised Language Learning: Representation Learning for NLP Distributional semantics Distributional hypothesis You shall know a word by the company it keeps (Firth) The meaning of a word is defined by the way it is used (Wittgenstein). it was authentic scrumpy, rather sharp and very strong we could taste a famous local product — scrumpy spending hours in the pub drinking scrumpy Cornish Scrumpy Medium Dry. £19.28 - Case

  19. Unsupervised Language Learning: Representation Learning for NLP Distributional semantics Distributional hypothesis You shall know a word by the company it keeps (Firth) The meaning of a word is defined by the way it is used (Wittgenstein). it was authentic scrumpy, rather sharp and very strong we could taste a famous local product — scrumpy spending hours in the pub drinking scrumpy Cornish Scrumpy Medium Dry. £19.28 - Case

  20. Unsupervised Language Learning: Representation Learning for NLP Distributional semantics Distributional hypothesis You shall know a word by the company it keeps (Firth) The meaning of a word is defined by the way it is used (Wittgenstein). it was authentic scrumpy, rather sharp and very strong we could taste a famous local product — scrumpy spending hours in the pub drinking scrumpy Cornish Scrumpy Medium Dry. £19.28 - Case

  21. Unsupervised Language Learning: Representation Learning for NLP Distributional semantics Distributional hypothesis You shall know a word by the company it keeps (Firth) The meaning of a word is defined by the way it is used (Wittgenstein). it was authentic scrumpy, rather sharp and very strong we could taste a famous local product — scrumpy spending hours in the pub drinking scrumpy Cornish Scrumpy Medium Dry. £19.28 - Case

  22. Unsupervised Language Learning: Representation Learning for NLP Distributional semantics Scrumpy

  23. Unsupervised Language Learning: Representation Learning for NLP Distributional semantics Distributional hypothesis This leads to the distributional hypothesis about word meaning: ◮ the context surrounding a given word provides information about its meaning; ◮ words are similar if they share similar linguistic contexts; ◮ semantic similarity ≈ distributional similarity.

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

UNSUPERVISED LEARNING IN R Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of clustering finding groups of homogeneous items Next up, dimensionality reduction Find structure in features

904 views • 29 slides
Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation

Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation

Code & Data : github.com/snakeztc/NeuralDialog-LAED Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation Tiancheng Zhao, Kyusong Lee and Maxine Eskenazi Language Technologies Institute, Carnegie

730 views • 28 slides
Unsupervised Visual Representation Learning by Context Prediction Berkan Demirel Most slides in

Unsupervised Visual Representation Learning by Context Prediction Berkan Demirel Most slides in

Unsupervised Visual Representation Learning by Context Prediction Berkan Demirel Most slides in this representation are adopted from authors' original presentation at ICCV 2015 ImageNet + Deep Learning Beagle - Image Retrieval - Detection

2.18k views • 33 slides
Unsupervised Learning of the Morphology of a Natural Language John Goldsmith University of

Unsupervised Learning of the Morphology of a Natural Language John Goldsmith University of

Unsupervised Learning of the Morphology of a Natural Language John Goldsmith University of Chicago Abstract This study reports the results of using Minimum Description Length analysis to model unsupervised learning of the morphology of

1.15k views • 101 slides
12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Wanli Ouyang, Zsolt Kira, Lawrence Neal, Raymond Yeh, Junting Lou and Teck-Yian Lim Unsupervised Learning in General Unsupervised learning is

1.48k views • 115 slides
Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised

Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised

Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised Learning Unsupervised Learning Setting Supervised learning requires the availability of labelled examples Labelling examples can be an extremely

752 views • 27 slides
Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on supervised learning methods such as regression and classification. In that setting we observe both a set of features X 1 , X 2 , . . . , X p for each

934 views • 50 slides
Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on supervised learning methods such as regression and classification. In that setting we observe both a set of features X 1 , X 2 , . . . , X p for each

721 views • 60 slides
Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no output classifications (labels) l Clustering is an important type of unsupervised learning PCA was another type of unsupervised learning l The

858 views • 38 slides
Unit 2: Natural Language Learning Unsupervised Learning (EM, forward-backward, inside-outside)

Unit 2: Natural Language Learning Unsupervised Learning (EM, forward-backward, inside-outside)

Natural Language Processing Spring 2017 Unit 2: Natural Language Learning Unsupervised Learning (EM, forward-backward, inside-outside) Liang Huang liang.huang.sh@gmAYl.com Review of Noisy-Channel Model CS 562 - EM 2 Example 1:

469 views • 23 slides
Language and Computers Unsupervised Learning Features & Classifying Documents Evidence

Language and Computers Unsupervised Learning Features & Classifying Documents Evidence

Language and Computers Classifying Documents Introduction Language Identification Machine Learning Supervised Learning Language and Computers Unsupervised Learning Features & Classifying Documents Evidence Measuring sucess

800 views • 45 slides
Unsupervised Learning of Visual Representation by Solving Jigsaw Puzzles, ECCV 16 2018/11/27

Unsupervised Learning of Visual Representation by Solving Jigsaw Puzzles, ECCV 16 2018/11/27

Unsupervised Learning of Visual Representation by Solving Jigsaw Puzzles, ECCV 16 2018/11/27 20173130 Jaeyoon Kim CS688 Paper Presentation Image Retrieval with Mixed initiative and Multimodal Feedback, BMVC 18 The system based on

336 views • 18 slides
An Unsupervised Autoregressive Model for Speech Representation Learning Yu-An Chung Wei-Ning Hsu

An Unsupervised Autoregressive Model for Speech Representation Learning Yu-An Chung Wei-Ning Hsu

An Unsupervised Autoregressive Model for Speech Representation Learning Yu-An Chung Wei-Ning Hsu Hao Tang James Glass Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology Cambridge, Massachusetts,

1.24k views • 13 slides
Stable and Efficient Representation Learning with Nonnegativity Constraints Tsung-Han Lin and

Stable and Efficient Representation Learning with Nonnegativity Constraints Tsung-Han Lin and

Stable and Efficient Representation Learning with Nonnegativity Constraints Tsung-Han Lin and H.T. Kung Unsupervised Representation Learning Layer 3 Representation Layer 2 Representation Encoding Sparse encoder Layer 1 Representation

304 views • 18 slides
Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain Sciences University of Trento 1 Unsupervised learning In unsupervised learning, we learn without training data. The idea is to find a

998 views • 67 slides
Lecture 8: Distributional semantics Models Getting distributions from text Real distributions

Lecture 8: Distributional semantics Models Getting distributions from text Real distributions

Natural Language Processing: Part II Overview of Natural Language Processing (L90): ACS Lecture 8: Distributional semantics Models Getting distributions from text Real distributions Similarity Distributions and classic lexical semantic

875 views • 56 slides
Aubergine Cobalt Navy Black

Aubergine Cobalt Navy Black

Aubergine Cobalt Navy Black

350 views • 4 slides
http://lcl.uniroma1.it ERC Starting Grant MultiJEDI No. 259234 ERC StG: Multilingual Joint

http://lcl.uniroma1.it ERC Starting Grant MultiJEDI No. 259234 ERC StG: Multilingual Joint

Babelfying the Multilingual Web: state-of-the-art Disambiguation and Entity Linking of Web pages in 50 languages Roberto Navigli navigli@di.uniroma1.it http://lcl.uniroma1.it ERC Starting Grant MultiJEDI No. 259234 ERC StG: Multilingual

1.16k views • 38 slides
Darrell Bethea June 8, 2011 Program 4 due Friday Final exam Comprehensive Monday,

Darrell Bethea June 8, 2011 Program 4 due Friday Final exam Comprehensive Monday,

Darrell Bethea June 8, 2011 Program 4 due Friday Final exam Comprehensive Monday, 6/13, 8-11 AM SN014 2 3 Inheritance and polymorphism 4 Person has a jump method, so all subclasses have a jump method Person Athlete

632 views • 25 slides
Exceptions Murray Cole Computer Science School of Informatics The University of Edinburgh

Exceptions Murray Cole Computer Science School of Informatics The University of Edinburgh

V I N E U R S E I H T Y T O H F G R E U D B I N Exceptions Murray Cole Computer Science School of Informatics The University of Edinburgh 1 Stuff that goes wrong Programming the wrong thing ! Programming the

448 views • 20 slides
Synonymy in an approach to combined distributional and compositional semantics Ann Copestake and

Synonymy in an approach to combined distributional and compositional semantics Ann Copestake and

Synonymy in an approach to combined distributional and compositional semantics Synonymy in an approach to combined distributional and compositional semantics Ann Copestake and Aurlie Herbelot Computer Laboratory University of Cambridge

736 views • 21 slides
Random tilings and representations of classical Lie groups Alexey Bufetov University of Bonn 21

Random tilings and representations of classical Lie groups Alexey Bufetov University of Bonn 21

Random tilings and representations of classical Lie groups Alexey Bufetov University of Bonn 21 November, 2019 Domino tilings of Aztec diamond The Aztec diamond of size n is the set of all lattice squares which are (fully) contained in { ( x ,

568 views • 31 slides
The Canadian Experience in Improving Access to Care HISA Aug. 1, 2018 Skill Testing Question

The Canadian Experience in Improving Access to Care HISA Aug. 1, 2018 Skill Testing Question

Joh ohn n Sin incla clair ir Ch Chie ief f Ex Executiv ecutive The Canadian Experience in Improving Access to Care HISA Aug. 1, 2018 Skill Testing Question Canada ranked 10 th out of 11 Countries for Access to Care Source, K.

545 views • 32 slides

More recommend