low dimensional embeddings of logic
play

Low-dimensional Embeddings of Logic aschel, 1 Matko Bosnjak, 1 Sameer - PowerPoint PPT Presentation

Sep 13, 2023 •333 likes •903 views

Low-dimensional Embeddings of Logic aschel, 1 Matko Bosnjak, 1 Sameer Singh 2 and Sebastian Riedel 1 Tim Rockt 1 University College London 2 University of Washington Machine Reading ACL 2014 Workshop on Semantic Parsing 26th June 2014

  1. Low-dimensional Embeddings of Logic aschel, 1 Matko Bosnjak, 1 Sameer Singh 2 and Sebastian Riedel 1 Tim Rockt¨ 1 University College London 2 University of Washington Machine Reading ACL 2014 Workshop on Semantic Parsing 26th June 2014

  2. Motivation Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  3. Motivation Machine Reading and Reasoning Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  4. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  5. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  6. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  7. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  8. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  9. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding Endothermic mammals that are found in Colugo Mammal Southeast Asia.” Kagu Ectothermic mammal ( Colugo ) HasFeathers “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  10. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding Endothermic mammals that are found in Colugo Mammal Southeast Asia.” Kagu Ectothermic mammal ( Colugo ) HasFeathers “All mammals are endothermic.” Wrong Predictions ∀ x : mammal ( x ) ⇒ endothermic ( x ) mammal ( Kagu ) Reasoning... ectothermic ( Colugo ) endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  11. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding Endothermic mammals that are found in Colugo Mammal Southeast Asia.” Kagu Ectothermic mammal ( Colugo ) HasFeathers “All mammals are endothermic.” Wrong Predictions ∀ x : mammal ( x ) ⇒ endothermic ( x ) mammal ( Kagu ) Reasoning... ectothermic ( Colugo ) endothermic ( Colugo ) I wish I had a distributed model... I wish I could fix this with... ∀ x : hasFeathers ( x ) ⇒ ¬ mammal ( x ) Mammal Endothermic ∀ x : animal ( x ) ⇒ Colugo endothermic ( x ) ⊕ ectothermic ( x ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  12. Information Extraction Evidence DB Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  13. Information Extraction Logic Evidence Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  14. Information Extraction Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  15. Information Extraction Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Background Knowledge Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  16. Logical Inference Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  17. Logical Inference Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), weight 0.53 DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  18. Inference via Distributed Representations Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013) Relation Tapir Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo endothermic DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  19. Inference via Distributed Representations Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) OpenIE Tapir mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  20. Inference via Distributed Representations Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) OpenIE Tapir mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  21. Distributed Representations that Simulate First-order Logical Reasoning Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) This Work OpenIE Tapir mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  22. Distributed Representations that Simulate First-order Logical Reasoning Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) This Work OpenIE mammal mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) Colugo endothermic(Colugo), Tapir “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  23. Distributed Representations that Simulate First-order Logical Reasoning Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) This Work OpenIE mammal mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) Colugo endothermic(Colugo), Tapir “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  24. Propositional Logic Logic Logical Tensor Calculus (Grefenstette, 2013) � 1 � 0 � � [true]; [false] ; 0 1

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings Agenda Traditional NLP Word Embeddings-1 Word Embeddings-2 Text preprocessing Topic Modeling ELMo Bag-of-words model Neural

900 views • 64 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

1.25k views • 75 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

2k views • 24 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

422 views • 17 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

828 views • 15 slides
Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme: Index dun mot: w i Table dembeddings (lookup matrix): V Embedding: e i e i = V( w i ) Reprsentation dun mot

780 views • 47 slides
Categories, Logic, Physics Higher Dimensional Birmingham Group Theory? September 21,

Categories, Logic, Physics Higher Dimensional Birmingham Group Theory? September 21,

What is and what should be Categories, Logic, Physics Higher Dimensional Birmingham Group Theory? September 21, 2010 Ronnie Brown What it should be Dreams shattered! Back to basics Groupoids to the rescue Homotopy What is

575 views • 38 slides
Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds University of California, San Diego Southern California Machine Learning Symposium, Caltech, 11/18/2018 Word Embeddings Language models which learn

537 views • 32 slides
What is best for spoken langage understanding: small but task-dependent embeddings or huge but

What is best for spoken langage understanding: small but task-dependent embeddings or huge but

What is best for spoken langage understanding: small but task-dependent embeddings or huge but out-of-domain embeddings Sahar Ghannay, Antoine Neuraz, Sophie Rosset 1 Goal Focus on semantic evaluation of common word embeddings

360 views • 16 slides
Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff

Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff

Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff and Qin Zhang IBM Research Almaden COLT13, June 12, 2013 1-1 Subspace embeddings Subspace embeddings: A distribution over linear maps : R n

810 views • 44 slides
Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an

Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an

Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an Embedding ? 3 Why Embeddings ? 4 Embeddings Pipeline 5 Whats Next Agenda 1 Team 2 Whats an Embedding ? 3 Why Embeddings ? 4

732 views • 40 slides
Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview Word types and tokens Training contextual embeddings Embeddings from Language Models (ELMo) 1 Overview Word types and tokens Training

670 views • 40 slides
Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments

Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments

Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl June 12, 2018 Drawings of Graphs Drawings of Graphs G = ( V, E ) is a simple graph . The set of vertices V is finite

596 views • 56 slides
What can we say about high- dimensional objects from a low-dimensional representation? 2

What can we say about high- dimensional objects from a low-dimensional representation? 2

Dimension Reduction with Certainty Rasmus Pagh IT University of Copenhagen S CALABLE S IMILARITY ECML-PKDD S EARCH September 21, 2016 Slides: goo.gl/hZoRWo 1 What can we say about high- dimensional objects from a low-dimensional

940 views • 75 slides
CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu

CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu

CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu Research Engineer, Facebook AI ledell@fb.com 1 Outline Word Embeddings word2vec Graph Embeddings Applications world2vec

769 views • 41 slides
Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to generate vector embeddings? One approach: feedforward neural language models Training a neural language model just to get word embeddings is expensive!

537 views • 35 slides
Project Proposals Reid Holmes Proposal Pitches Buy My Book Hack UW Flow Kluster

Project Proposals Reid Holmes Proposal Pitches Buy My Book Hack UW Flow Kluster

Project Proposals Reid Holmes Proposal Pitches Buy My Book Hack UW Flow Kluster MoPhoCon Eat Out Radius Spaceman Broken Draw REID HOLMES - SE2: SOFTWARE DESIGN & ARCHITECTURE Buy My Book REID HOLMES -

532 views • 39 slides
Recommender Systems Class Data Mining Program Master in Computer Engineering University

Recommender Systems Class Data Mining Program Master in Computer Engineering University

Recommender Systems Class Data Mining Program Master in Computer Engineering University Sapienza University of Rome Semester Fall 2017 Slides by Carlos Castillo http://chato.cl/ Sources: Ricci, Rokach and Shapira: Introduction to

957 views • 67 slides
Ubiquitous and Mobile Computing CS 528: MobileMiner Mining Your Frequent Behavior Patterns on Your

Ubiquitous and Mobile Computing CS 528: MobileMiner Mining Your Frequent Behavior Patterns on Your

Ubiquitous and Mobile Computing CS 528: MobileMiner Mining Your Frequent Behavior Patterns on Your Phone Muxi Qi Electrical and Computer Engineering Dept. Worcester Polytechnic Institute (WPI) OUTLINE Introduction System Design

696 views • 33 slides
Ubiquitous Video W3C Video on the Web Workshop 20071212 B. Fairman 1 Sony US Advanced

Ubiquitous Video W3C Video on the Web Workshop 20071212 B. Fairman 1 Sony US Advanced

Ubiquitous Video W3C Video on the Web Workshop 20071212 B. Fairman 1 Sony US Advanced Technologies Center 20071212 Sony Position for the W3C Video on the Web Workshop Sony is very active in standardization areas that are complimentary

336 views • 9 slides
THE (UN)DESERVING RICH American Beliefs about Income Inequality from 1980 to the Occupy Wall

THE (UN)DESERVING RICH American Beliefs about Income Inequality from 1980 to the Occupy Wall

THE (UN)DESERVING RICH American Beliefs about Income Inequality from 1980 to the Occupy Wall Street Movement Spring, 2012 Leslie McCall Department of Sociology Faculty Fellow, Institute for Policy Research Northwestern University The

644 views • 37 slides
Java SE 9 and the Application Server Kevin Sutter MicroProfile and Java EE Architect

Java SE 9 and the Application Server Kevin Sutter MicroProfile and Java EE Architect

InterConnect EclipseCon Europe 2017 2017 Java SE 9 and the Application Server Kevin Sutter MicroProfile and Java EE Architect @kwsutter 1 Java SE 9 Standalone 2 10/30/17 Java 9 Standard Features JSR 379: Java SE 9 Release

468 views • 32 slides
Control Department of Chemical Engineering I.I.T. Bombay, India Effect of disturbances in

Control Department of Chemical Engineering I.I.T. Bombay, India Effect of disturbances in

Department of Chemical Engineering I.I.T. Bombay, India Lecture 18 : Cascade and Feedforward Control Department of Chemical Engineering I.I.T. Bombay, India Effect of disturbances in feedback control d 1 d 2 y d + y C P - Traditional

758 views • 14 slides
Evaluating Machine Learned User Experiences Asela Gunawardana Intelligent User Experiences

Evaluating Machine Learned User Experiences Asela Gunawardana Intelligent User Experiences

Evaluating Machine Learned User Experiences Asela Gunawardana Intelligent User Experiences Microsoft Research The typical machine learning problem ,

560 views • 45 slides

More recommend