gaussian mixture latent vector grammars
play

Gaussian Mixture Latent Vector Grammars Yanpeng Zhao Liwen Zhang - PowerPoint PPT Presentation

Aug 14, 2022 •114 likes •422 views

Gaussian Mixture Latent Vector Grammars Yanpeng Zhao Liwen Zhang Kewei Tu School of Information Science and Technology ShanghaiTech University ACL 2018 PCFG Parsing & Limitations Constituency Parsing Grammars Prob.

  1. Gaussian Mixture Latent Vector Grammars Yanpeng Zhao Liwen Zhang Kewei Tu School of Information Science and Technology ShanghaiTech University ACL 2018

  2. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … � 2

  3. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … S T 1 = NP VP P V NP P He found me � 3

  4. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … S S T 1 = T 2 = NP VP NP VP P V NP P V NP P P He found me me found He � 4

  5. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … S S P ( T 1 ) = P ( T 2 ) T 1 = T 2 = NP VP NP VP P V NP P V NP He found me 6 = me found He P P limitations He found me me found He � 5

  6. Tree Annotation & Lexicalization SˆROOT S-found NPˆS VPˆS-BD-Z NP-He VP-found P-Z V-BD-Z NPˆVP-O P-He V-found NP-me P-O P-me He found me He found me [Johnson. 1998; Klein et al. 2003] [Collins. 1997; Charniak. 2000] Tree Annotation Lexicalization � 6

  7. Latent Variable Grammars (LVGs) [Matsuzaki et al. 2005; Petrov et al. 2007] S[ x 1 ] S 0 NP[ x 2 ] VP[ x 4 ] NP 2 VP 0 P[ x 3 ] V[ x 5 ] NP[ x 6 ] P 0 V 1 NP 2 P[ x 7 ] P 3 He found me He found me Annotated parse tree A subtype parse tree • Discrete latent variables x 1 , x 2 , x 3 . . . • Model a finite number of nonterminal subtypes � 7

  8. Refining Syntactic Category SˆROOT S-found S[ x 1 ] NPˆS VPˆS-BD-Z NP-He VP-found NP[ x 2 ] VP[ x 4 ] P-Z V-BD-Z NPˆVP-O P-He V-found NP-me P[ x 3 ] V[ x 5 ] NP[ x 6 ] P-O P-me P[ x 7 ] He found me He found me He found me Tree annotation Lexicalization LVGs [Klein et al. 2003] [Charniak. 2000] [Petrov et al. 2007] F1: 85.7 F1: 89.6 F1: 90.1 � 8

  9. Latent Vector Grammars (LVeGs) S[ x 1 ] S [0 . 05 , 0 . 2] NP[ x 2 ] VP[ x 4 ] NP [0 . 4 , 1 . 7] VP [3 . 4 , 0 . 9] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P [0 . 3 , 2 . 1] V [0 . 1 , 0 . 2] NP [1 . 3 , 0 . 7] P[ x 7 ] P [0 . 5 , 1 . 4] He found me He found me Annotated parse tree A subtype parse tree • Continuous latent vectors x 1 , x 2 , x 3 . . . • Model an infinite number of nonterminal subtypes � 9

  10. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me � 10

  11. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me Discrete latent variables Continuous latent vectors x 1 , x 2 , x 3 . . . x 1 , x 2 , x 3 . . . Annotated rule NP[ x 2 ] � P[ x 3 ] Annotated rule NP[ x 2 ] � P[ x 3 ] � 11

  12. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me Discrete latent variables Continuous latent vectors x 1 , x 2 , x 3 . . . x 1 , x 2 , x 3 . . . Annotated rule Annotated rule NP[ x 2 ] � P[ x 3 ] NP[ x 2 ] � P[ x 3 ] A finite set of subtype rules An infinite set of subtype rules Subtype rule Subtype rule NP [2 . 3 , 1 . 7] � P [0 . 4 , 1 . 3] NP 0 � P 1 � 12

  13. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me Discrete latent variables Continuous latent vectors x 1 , x 2 , x 3 . . . x 1 , x 2 , x 3 . . . Annotated rule Annotated rule NP[ x 2 ] � P[ x 3 ] NP[ x 2 ] � P[ x 3 ] A finite set of subtype rules An infinite set of subtype rules Subtype rule Subtype rule NP [2 . 3 , 1 . 7] � P [0 . 4 , 1 . 3] NP 0 � P 1 Prob. of the subtype rule: Weight density of the subtype rule: W NP � P ( x 2 = [2 . 3 , 1 . 7] , x 3 = [0 . 4 , 1 . 3]) W NP � P ( x 2 = 0 , x 3 = 1) � 13

  14. LVGs as Special Case of LVeGs S[ x 1 ] Discrete Variables NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] One-hot Vectors He found me � 14

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Kernel Methods for Regression Support Vector Regression Gaussian Mixture Regression Gaussian

Kernel Methods for Regression Support Vector Regression Gaussian Mixture Regression Gaussian

MACHINE LEARNING MACHINE LEARNING MACHINE LEARNING Kernel Methods for Regression Support Vector Regression Gaussian Mixture Regression Gaussian Process Regression 1 MACHINE LEARNING 2012 MACHINE LEARNING MACHINE LEARNING Problem

945 views • 71 slides
MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy

MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy

E9 205 Machine Learning for Signal Processing MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy (sriramg@iisc.ac.in) Teaching Assistant - Prachi Singh (prachisingh@iisc.ac.in). Finding the parameters of the

390 views • 14 slides
Multivariate Gaussian Mean vector: Covariance matrix: 2 1 Conditioning a Gaussian Joint

Multivariate Gaussian Mean vector: Covariance matrix: 2 1 Conditioning a Gaussian Joint

Readings: K&F: 6.1, 6.2, 6.3, 14.1, 14.2, 14.3, 14.4, Kalman Filters Gaussian MNs Graphical Models 10708 Carlos Guestrin Carnegie Mellon University December 1 st , 2008 1 Multivariate Gaussian Mean vector: Covariance matrix: 2 1

257 views • 11 slides
Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Mixture Models: definition Mixture models: Linear supper-position of mixtures or components | =

1.04k views • 39 slides
Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline

Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline

Introduction Using Gaussian Mixture Model to Detect Figurative Language Evaluating the GMM Approach Conclusion Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline Sporleder Cluster of Excellence, MMCI

411 views • 21 slides
Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations BIOSTAT830: Graphical Models December 08, 2016 Introduction - INLA Inference for latent Gaussian Markov random field

605 views • 35 slides
Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff

Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff

Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff McLachlan (University of Queensland, Australia) JOCLAD 2018, Lisbona, April 5th, 2018 Outline Deep Learning Mixture Models Deep Gaussian Mixture

915 views • 66 slides
Applied Machine Learning Expectation Maximization for Mixture of Gaussians Siamak Ravanbakhsh

Applied Machine Learning Expectation Maximization for Mixture of Gaussians Siamak Ravanbakhsh

Applied Machine Learning Expectation Maximization for Mixture of Gaussians Siamak Ravanbakhsh COMP 551 (Fall 2020) Learning objectives what is a latent variable model? Gaussian mixture model the intuition behind the Expectation-Maximization

530 views • 18 slides
Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models Yarin Gal Mark van der Wilk Carl E. Rasmussen yg279@cam.ac.uk June 25th, 2014 Outline Gaussian process regression and latent variable

1.18k views • 54 slides
Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan

Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan

Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan Joint work with Benjamin Marlin, and Kevin Murphy University of British Columbia June 29, 2011 Piecewise Bounds for Binary Latent Gaussian Models

694 views • 34 slides
E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes Mauricio A. Zhenwen Dai Alvarez Neil D. Lawrence Gaussian Process Approximation Workshop, 2017 Motivation I Machine learning has been very

685 views • 21 slides
The Structure of the Worst Noise in Gaussian Vector Broadcast Channels Wei Yu University of

The Structure of the Worst Noise in Gaussian Vector Broadcast Channels Wei Yu University of

The Structure of the Worst Noise in Gaussian Vector Broadcast Channels Wei Yu University of Toronto March 19, 2003 DIMACS Workshop on Network Information Theory Outline Sum capacity of Gaussian vector broadcast channels. Complete

554 views • 38 slides
Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative

Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative

Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative Sequence Training Chao Zhang and Phil Woodland March 8, 2017 Cambridge University Engineering Department Introduction Tandem Systems as Mixture

926 views • 17 slides
Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen Tannenbaum Presented by: Zach Lucas Intro and Motivation A mixture model is a probabilistic model describing properties of populations with

357 views • 35 slides
Latent class analysis and finite mixture models with Stata Isabel Canette Principal

Latent class analysis and finite mixture models with Stata Isabel Canette Principal

Latent class analysis and finite mixture models with Stata Isabel Canette Principal Mathematician and Statistician StataCorp LLC 2017 Stata Users Group Meeting Madrid, October 19th, 2017 Introduction Latent class analysis (LCA)

658 views • 32 slides
Parsing with Compositional Vector Grammars BY RICHARD SOCHER, JOHN BAUER, CHRISTOPHER D. MANNING,

Parsing with Compositional Vector Grammars BY RICHARD SOCHER, JOHN BAUER, CHRISTOPHER D. MANNING,

Parsing with Compositional Vector Grammars BY RICHARD SOCHER, JOHN BAUER, CHRISTOPHER D. MANNING, ANDREW Y. NG PRESENT BY YUNCHENG WU Outline Introduction Related Works Compositional Vector Grammars (this paper) Experiments Takeaways

888 views • 34 slides
ZooKeeper Wait-free coordination for Internet-scale systems Patrick Hunt and Mahadev (Yahoo!

ZooKeeper Wait-free coordination for Internet-scale systems Patrick Hunt and Mahadev (Yahoo!

ZooKeeper Wait-free coordination for Internet-scale systems Patrick Hunt and Mahadev (Yahoo! Grid) Flavio Junqueira and Benjamin Reed (Yahoo! Research) Internet-scale Challenges Lots of servers, users, data FLP, CAP Mere mortal

271 views • 24 slides
Public Self-consciousness for Endowing Dialogue Agents with Consistent Persona 2020 BAICS workshop

Public Self-consciousness for Endowing Dialogue Agents with Consistent Persona 2020 BAICS workshop

Public Self-consciousness for Endowing Dialogue Agents with Consistent Persona 2020 BAICS workshop (Oral) Hyunwoo Kim Byeongchang Kim Gunhee Kim V I S I O N & L E A R N I N G L A B SEOUL NATIONAL UNIVERSITY The Consistency Problem

714 views • 37 slides
Security 2 CS 4410 Operating Systems [E. Birrell, A. Bracy, F. B. Schneider, E. Sirer, R. Van

Security 2 CS 4410 Operating Systems [E. Birrell, A. Bracy, F. B. Schneider, E. Sirer, R. Van

Security 2 CS 4410 Operating Systems [E. Birrell, A. Bracy, F. B. Schneider, E. Sirer, R. Van Renesse] References: Security Introduction and Access Control by Fred B. Schneider Security in Computer Systems Gold (Au) Standard for Security

610 views • 24 slides
Access Control Lists Don Porter CSE 506 Background (1) If everything in Unix is a file

Access Control Lists Don Porter CSE 506 Background (1) If everything in Unix is a file

Access Control Lists Don Porter CSE 506 Background (1) If everything in Unix is a file Everything in Windows is an object Why not files? Not all OS abstractions make sense as a file Examples: Eject button on an optical drive

908 views • 22 slides
LaSEWeb : Automating Search Strategies over Semi-Structured Web Data Oleksandr Polozov Sumit

LaSEWeb : Automating Search Strategies over Semi-Structured Web Data Oleksandr Polozov Sumit

LaSEWeb : Automating Search Strategies over Semi-Structured Web Data Oleksandr Polozov Sumit Gulwani University of Washington Microsoft Research polozov@cs.washington.edu sumitg@microsoft.com KDD 2014 August 27, 2014 Motivation: search

478 views • 25 slides
More than just Load Balancing iRODS Using HAProxy Tony Edgin iRODS UGM 2019 Purpose Previous

More than just Load Balancing iRODS Using HAProxy Tony Edgin iRODS UGM 2019 Purpose Previous

Transforming Science Through Data-driven Discovery More than just Load Balancing iRODS Using HAProxy Tony Edgin iRODS UGM 2019 Purpose Previous work shows how to proxy iRODS for high availability. High Availability iRODS System (HAIRS)

424 views • 19 slides
Making Linux Protection Mechanisms Egalitarian with UserFS Taesoo Kim and Nickolai Zeldovich

Making Linux Protection Mechanisms Egalitarian with UserFS Taesoo Kim and Nickolai Zeldovich

Making Linux Protection Mechanisms Egalitarian with UserFS Taesoo Kim and Nickolai Zeldovich MIT CSAIL Overview: How to build secure applications? Simple in principle : - reduce privileges of application components - enforce policy at

600 views • 58 slides
Advertisement for ACL Workshops Workshop on Narrative Understanding, Workshop on Neural

Advertisement for ACL Workshops Workshop on Narrative Understanding, Workshop on Neural

Advertisement for ACL Workshops Workshop on Narrative Understanding, Workshop on Neural Generation and Storylines, and Events (NUSE) Translation We solicit papers related to narrative Topics of interest include neural models for understanding

688 views • 29 slides

More recommend