reified context models
play

Reified Context Models Jacob Steinhardt Percy Liang Stanford - PowerPoint PPT Presentation

Jan 20, 2024 •542 likes •1.16k views

Reified Context Models Jacob Steinhardt Percy Liang Stanford University { jsteinhardt,pliang } @cs.stanford.edu July 8, 2015 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 1 / 11 Structured Prediction Task input

  1. Reified Context Models Jacob Steinhardt Percy Liang Stanford University { jsteinhardt,pliang } @cs.stanford.edu July 8, 2015 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 1 / 11

  2. Structured Prediction Task input x : output y : v o l c a n i c J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 2 / 11

  3. Contexts Are Key v o c a l J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 3 / 11

  4. Contexts Are Key v o c a l v *o ***c DP: **l J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 3 / 11

  5. Contexts Are Key v o c a l v *o ***c DP: **l v o c a l v vo beam search: vol volc J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 3 / 11

  6. Contexts Are Key v o c a l v *o ***c DP: **l v o c a l v vo beam search: vol volc Key idea: contexts!   ao       bo   *o def = co     .   .   . J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 3 / 11

  7. Desiderata r *o **l ***c v *a **i ***r coverage (short contexts) better uncertainty estimates (precision) stabler partially supervised learning updates J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 4 / 11

  8. Desiderata r *o **l ***c v *a **i ***r coverage (short contexts) better uncertainty estimates (precision) stabler partially supervised learning updates J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 4 / 11

  9. Desiderata r *o **l ***c v *a **i ***r coverage (short contexts) better uncertainty estimates (precision) stabler partially supervised learning updates J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 4 / 11

  10. Desiderata r *o **l ***c v *a **i ***r coverage (short contexts) better uncertainty estimates (precision) stabler partially supervised learning updates r ro rol rolc v ra ral ralc expressivity (long contexts) capture complex dependencies J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 4 / 11

  11. Desiderata r *o **l ***c v *a **i ***r coverage (short contexts) better uncertainty estimates (precision) stabler partially supervised learning updates r ro rol rolc v ra ral ralc expressivity (long contexts) capture complex dependencies J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 4 / 11

  12. Desiderata r *o **l ***c v *a **i ***r coverage (short contexts) better uncertainty estimates (precision) stabler partially supervised learning updates r ro rol rolc v ra ral ralc expressivity (long contexts) capture complex dependencies J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 4 / 11

  13. Desiderata r *o **l ***c v *a **i ***r coverage (short contexts) better uncertainty estimates (precision) stabler partially supervised learning updates r ro rol rolc v ra ral ralc expressivity (long contexts) capture complex dependencies r ro rol *olc v ra ral ***c ← best of both worlds y *o *ol ***r * ** *** **** J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 4 / 11

  14. Reifying Contexts input x : J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 5 / 11

  15. Reifying Contexts input x : output y : v o l c a n i c J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 5 / 11

  16. Reifying Contexts input x : output y : v o l c a n i c context c : v *o *ol *olc ······ J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 5 / 11

  17. Reifying Contexts input x : output y : v o l c a n i c context c : v *o *ol *olc ······ r ro rol *olc v ra ral ***c y *o *ol ***r * ** *** **** J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 5 / 11

  18. Reifying Contexts input x : output y : v o l c a n i c context c : v *o *ol *olc ······ r ro rol *olc v ra ral ***c ← “context sets” y *o *ol ***r * ** *** **** C 1 C 2 C 3 C 4 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 5 / 11

  19. Reifying Contexts input x : output y : v o l c a n i c context c : v *o *ol *olc ······ r ro rol *olc v ra ral ***c ← “context sets” y *o *ol ***r * ** *** **** C 1 C 2 C 3 C 4 Challenge: how to trade off contexts of different lengths? J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 5 / 11

  20. Reifying Contexts input x : output y : v o l c a n i c context c : v *o *ol *olc ······ r ro rol *olc v ra ral ***c ← “context sets” y *o *ol ***r * ** *** **** C 1 C 2 C 3 C 4 Challenge: how to trade off contexts of different lengths? = ⇒ Reify contexts as part of model! J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 5 / 11

  21. Reified Context Models Given: context sets C 1 ,..., C L J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 6 / 11

  22. Reified Context Models Given: context sets C 1 ,..., C L features φ i ( c i − 1 , y i ) J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 6 / 11

  23. Reified Context Models Given: context sets C 1 ,..., C L features φ i ( c i − 1 , y i ) Define the model � � L θ ⊤ φ i ( c i − 1 , y i ) ∑ p θ ( y 1 : L , c 1 : L − 1 ) ∝ exp · κ ( y , c ) � �� � i = 1 consistency J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 6 / 11

  24. Reified Context Models Given: context sets C 1 ,..., C L features φ i ( c i − 1 , y i ) Define the model � � L θ ⊤ φ i ( c i − 1 , y i ) ∑ p θ ( y 1 : L , c 1 : L − 1 ) ∝ exp · κ ( y , c ) � �� � i = 1 consistency Graphical model structure: C 1 C 2 C 3 C 4 κ κ κ κ κ Y 1 Y 2 Y 3 Y 4 Y 5 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 6 / 11

  25. Reified Context Models Given: context sets C 1 ,..., C L features φ i ( c i − 1 , y i ) Define the model � � L θ ⊤ φ i ( c i − 1 , y i ) ∑ p θ ( y 1 : L , c 1 : L − 1 ) ∝ exp · κ ( y , c ) � �� � i = 1 consistency Graphical model structure: C 1 C 2 C 3 C 4 φ 1 φ 2 φ 3 φ 4 φ 5 Y 1 Y 2 Y 3 Y 4 Y 5 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 6 / 11

  26. Reified Context Models Given: context sets C 1 ,..., C L features φ i ( c i − 1 , y i ) Define the model � � L θ ⊤ φ i ( c i − 1 , y i ) ∑ p θ ( y 1 : L , c 1 : L − 1 ) ∝ exp · κ ( y , c ) � �� � i = 1 consistency Graphical model structure: C 1 C 2 C 3 C 4 inference via φ 1 φ 2 φ 3 φ 4 φ 5 forward-backward! Y 1 Y 2 Y 3 Y 4 Y 5 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 6 / 11

  27. Adaptive Context Selection Select context sets C i during forward pass of inference J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

  28. Adaptive Context Selection Select context sets C i during forward pass of inference Greedily select contexts with largest mass J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

  29. Adaptive Context Selection Select context sets C i during forward pass of inference Greedily select contexts with largest mass a b c d e . . . J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

  30. Adaptive Context Selection Select context sets C i during forward pass of inference Greedily select contexts with largest mass a b c c d e e . . . J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

  31. Adaptive Context Selection Select context sets C i during forward pass of inference Greedily select contexts with largest mass a c b c c e d ⋆ e e . . . C 1 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

  32. Adaptive Context Selection Select context sets C i during forward pass of inference Greedily select contexts with largest mass ca a cb . . c b . c c e ea d eb ⋆ . . e e . . . ⋆ a . . . . C 1 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

  33. Adaptive Context Selection Select context sets C i during forward pass of inference Greedily select contexts with largest mass ca ca a cb . . c b . c c e ea d eb ⋆ . . e e . . . ⋆ a ⋆ a . . . . C 1 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

  34. Adaptive Context Selection Select context sets C i during forward pass of inference Greedily select contexts with largest mass ca ca a cb . . c ca b . c c e ea ⋆ a d eb ⋆ ⋆⋆ . . e e . . . ⋆ a ⋆ a . . . . C 1 C 2 J. Steinhardt & P. Liang (Stanford) Reified Context Models July 8, 2015 7 / 11

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Formal Formal Component Component Models Models for for Context Context

Formal Formal Component Component Models Models for for Context Context

Formal Formal Component Component Models Models for for Context Context Awareness Awareness Deploy FP7 MatsNeovius andKaisaSere FMCO2008 boAkademi FacultyofTechnology, 1 22.10.2008

431 views • 22 slides
Practical reified trees (not only) for GPGPU @ochafik http://github.com/ochafik/Scalaxy

Practical reified trees (not only) for GPGPU @ochafik http://github.com/ochafik/Scalaxy

Practical reified trees (not only) for GPGPU @ochafik http://github.com/ochafik/Scalaxy http://github.com/ochafik/ScalaCL Who am I? Hobby Scala enthusiast for 4 years I hate technology boundaries ScalaCL: runs Scala on graphic

422 views • 25 slides
Existing Context models Torralba,*Sinha*(2001)* Carbone(o,*de*Freitas*&*Barnard*(2004)*

Existing Context models Torralba,*Sinha*(2001)* Carbone(o,*de*Freitas*&*Barnard*(2004)*

PanoContext A Whole-room 3D Context Model for Panoramic Scene Understanding by Yinda Zhang, Shuran Song, Ping Tan, Jianxiong Xiao Presented by: William Xie Existing Context models Torralba,*Sinha*(2001)*

1.22k views • 99 slides
Context aware factors in rearchitecting two-level models into multilevel models Mira Balaban,

Context aware factors in rearchitecting two-level models into multilevel models Mira Balaban,

Context aware factors in rearchitecting two-level models into multilevel models Mira Balaban, Igal Khitron, Azzam Maraee Ben-Gurion University Israel 1 MULTI 2018 October 2018 Essential MLM rearchitecting The type-Instance pattern and its

476 views • 18 slides
Models fitness -for-purpose in the context of exceedance modelling Jenny Stocker & Kate

Models fitness -for-purpose in the context of exceedance modelling Jenny Stocker & Kate

WG1 discussion topic: Models fitness -for-purpose in the context of exceedance modelling Jenny Stocker & Kate Johnson FAIRMODE Technical Meeting June 2017 Athens Greece Contents Context Specific questions: 1. Do you see

379 views • 13 slides
Context Change and Versatile Models in Machine Learning Jos Hernndez-Orallo Universitat

Context Change and Versatile Models in Machine Learning Jos Hernndez-Orallo Universitat

Context Change and Versatile Models in Machine Learning Jos Hernndez-Orallo Universitat Politcnica de Valncia jorallo@dsic.upv.es ECML Workshop on Learning over Multiple Contexts Nancy, 19 September 2014 Spot the difference CONTEXT

548 views • 32 slides
Context in business process models: What is the use? Dr.ir. Hajo Reijers www.reijers.com Focus

Context in business process models: What is the use? Dr.ir. Hajo Reijers www.reijers.com Focus

Context in business process models: What is the use? Dr.ir. Hajo Reijers www.reijers.com Focus of this talk Context awareness is hot: Mobile applications, web systems, pervasive computing Business process modeling is hot:

757 views • 30 slides
Drawing Drawing models Graphics context Display lists Painters Algorithm Clipping &

Drawing Drawing models Graphics context Display lists Painters Algorithm Clipping &

Drawing Drawing models Graphics context Display lists Painters Algorithm Clipping & Double-buffering 1 Drawing Primitives Three conceptual models for drawing: Pixel SetPixel(x, y, colour) DrawImage(x, y, w, h, img) Stroke

524 views • 23 slides
Drawing Drawing models Graphics context Display lists Painters Algorithm Clipping &

Drawing Drawing models Graphics context Display lists Painters Algorithm Clipping &

Drawing Drawing models Graphics context Display lists Painters Algorithm Clipping & Double-buffering 1 Drawing Primitives Three conceptual models for drawing: Pixel SetPixel(x, y, colour) DrawImage(x, y, w, h, img) Stroke

412 views • 19 slides
Lecture II: Neutrino Mass Models in Context M.J. Ramsey-Musolf U Mass Amherst

Lecture II: Neutrino Mass Models in Context M.J. Ramsey-Musolf U Mass Amherst

Lecture II: Neutrino Mass Models in Context M.J. Ramsey-Musolf U Mass Amherst http://www.physics.umass.edu/acfi/ ACFI NLDBD School 10/31-11/3 2017 1 Lecture II Goals Provide broader BSM context for 0 decay Provide a simple

1.17k views • 89 slides
Context, initial objectives, orientations Context (Ph.D.) Distant learning

Context, initial objectives, orientations Context (Ph.D.) Distant learning

Context, initial objectives, orientations Context (Ph.D.) Distant learning Instructional design models Exploring potential addons of UML (meta)-modeling for the design and implementation of co-operative Problem- Based Learning

562 views • 12 slides
SEARCH AND CONTEXT Susan Dumais, Microsoft Research Overview Importance of context in

SEARCH AND CONTEXT Susan Dumais, Microsoft Research Overview Importance of context in

SEARCH AND CONTEXT Susan Dumais, Microsoft Research Overview Importance of context in information retrieval Potential for personalization framework Examples with varied user models and evaluation methods Personal navigation

295 views • 26 slides
More Models of Computation Lecture 24 Context Free Grammars Circuits Decision Trees Branching

More Models of Computation Lecture 24 Context Free Grammars Circuits Decision Trees Branching

More Models of Computation Lecture 24 Context Free Grammars Circuits Decision Trees Branching Programs Context-Free Grammar Example: a (simplistic) syntax for arithmetic expressions Set of rewriting rules over Expr Expr + Expr

387 views • 15 slides
EVALUATION OF ATMOSPHERIC DISPERSION MODELS IN A RISK ASSESSMENT CONTEXT Bertrand Sapolin 1 ,

EVALUATION OF ATMOSPHERIC DISPERSION MODELS IN A RISK ASSESSMENT CONTEXT Bertrand Sapolin 1 ,

METHODOLOGY FOR STATISTICAL EVALUATION OF ATMOSPHERIC DISPERSION MODELS IN A RISK ASSESSMENT CONTEXT Bertrand Sapolin 1 , Gilles Bergametti 2 , Philippe Bouteilloux 1 , Alain Dutot 2 13 th International Conference on Harmonisation within

304 views • 17 slides
Presentation Practice education in the context of agency practice models: the implications for

Presentation Practice education in the context of agency practice models: the implications for

Presentation Practice education in the context of agency practice models: the implications for practice education and the commodification of social work knowledge . Tom Wilks Approaches to theory A specific theoretical orientation A

437 views • 20 slides
Reimagining Home Care Emerging models of care 24 February 2017 Ewan King, SCIE Introduction

Reimagining Home Care Emerging models of care 24 February 2017 Ewan King, SCIE Introduction

Reimagining Home Care Emerging models of care 24 February 2017 Ewan King, SCIE Introduction Context Emerging trends Promising models Tools and resources Context Pervasive underfunding of sector: 93 % of providers trading

733 views • 55 slides
Expressivity within second-order transitive-closure logic Jonni Virtema Transitive closure

Expressivity within second-order transitive-closure logic Jonni Virtema Transitive closure

Expressivity within second-order transitive-closure logic Expressivity within second-order transitive-closure logic Jonni Virtema Transitive closure FO(TC) & SO(TC) Jonni Virtema Examples Expressivity Hasselt University, Belgium

499 views • 35 slides
Expressivity within second-order transitive-closure logic Background Transitive closure FO(TC)

Expressivity within second-order transitive-closure logic Background Transitive closure FO(TC)

Expressivity within second-order transitive-closure logic Jonni Virtema Expressivity within second-order transitive-closure logic Background Transitive closure FO(TC) & SO(TC) Examples Jonni Virtema Expressivity Hasselt University,

804 views • 44 slides
Expressive curves Sergey Fomin University of Michigan arXiv:2006.14066 (with E. Shustin)

Expressive curves Sergey Fomin University of Michigan arXiv:2006.14066 (with E. Shustin)

Expressive curves Sergey Fomin University of Michigan arXiv:2006.14066 (with E. Shustin) arXiv:1711.10598 (with P. Pylyavskyy, E. Shustin, D. Thurston) Sergey Fomin (University of Michigan) Expressive curves August 4, 2020 1 / 28 Rolles

1.13k views • 47 slides
On the expressivity of total reversible programming languages Luca Paolini and Luca Roversi

On the expressivity of total reversible programming languages Luca Paolini and Luca Roversi

On the expressivity of total reversible programming languages Luca Paolini and Luca Roversi and Armando Matos Universit` a degli Studi di Torino Universidade do Porto July 2020 RC2020 1 / 11 Luca Paolini: On the

1.01k views • 21 slides
DeepProbLog: Neural Probabilistic Logic Programming Robin Manhaeve, Sebastijan Dumani ,

DeepProbLog: Neural Probabilistic Logic Programming Robin Manhaeve, Sebastijan Dumani ,

DeepProbLog: Neural Probabilistic Logic Programming Robin Manhaeve, Sebastijan Dumani , Angelika Kimmig, Thomas Demeester*, Luc De Raedt* * Joint last authors Real-life problems involve two important aspects. Sub-symbolic perception

552 views • 13 slides
Statistical mechanics of deep learning Surya Ganguli Dept. of Applied Physics, Neurobiology,

Statistical mechanics of deep learning Surya Ganguli Dept. of Applied Physics, Neurobiology,

Statistical mechanics of deep learning Surya Ganguli Dept. of Applied Physics, Neurobiology, and Electrical Engineering Stanford University NIH NIH Funding: Bio-X Neuroventures Bio-X Neuroventures Burroughs Burroughs Wellcome

3.2k views • 35 slides
On the Number of Linear Regions of Convolutional Neural Networks (joint with L. Huang, M. Yu, L.

On the Number of Linear Regions of Convolutional Neural Networks (joint with L. Huang, M. Yu, L.

On the Number of Linear Regions of Convolutional Neural Networks (joint with L. Huang, M. Yu, L. Liu, F . Zhu and L. Shao) Huan Xiong Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) ICML 2020 Huan Xiong Number of Linear

662 views • 10 slides
(LMCS, p. 317) V.1 FirstOrder Logic This is the most powerful, most expressive logic that we

(LMCS, p. 317) V.1 FirstOrder Logic This is the most powerful, most expressive logic that we

(LMCS, p. 317) V.1 FirstOrder Logic This is the most powerful, most expressive logic that we will examine. Our version of first-order logic will use the following symbols: variables connectives ( , , , , )

1.09k views • 97 slides

More recommend