reasoning with neural networks
play

Reasoning with Neural Networks Rodrigo Toro Icarte - PowerPoint PPT Presentation

Aug 14, 2022 •136 likes •750 views

University of Toronto Faculty of Arts and Science Department of Computer Science Reasoning with Neural Networks Rodrigo Toro Icarte (rntoro@cs.toronto.edu) March 08, 2016 Introduction Reasoning with Neural Networks Questions References

  1. University of Toronto Faculty of Arts and Science Department of Computer Science Reasoning with Neural Networks Rodrigo Toro Icarte (rntoro@cs.toronto.edu) March 08, 2016

  2. Introduction Reasoning with Neural Networks Questions References Motivation Could a crocodile run a steeplechase? 1 1 The example was borrowed from Levesque (2014)

  3. Introduction Reasoning with Neural Networks Questions References Symbolic approach KB : ... ∀ x.Crocodile ( x ) ⊃ WeakLegs ( x ) ... ∀ x.WeakLegs ( x ) ⊃ ¬ CanJump ( x ) ... ∀ x. ¬ CanJump ( x ) ⊃ ¬ CanSteeplechase ( x ) ... Query : ¬∃ x.Crocodile ( x ) ∧ CanSteeplechase ( x )

  4. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a )

  5. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a )

  6. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a ) Observations : - This is SLD resolution.

  7. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a ) Observations : - This is SLD resolution. - Scalability problems.

  8. Introduction Reasoning with Neural Networks Questions References A Neural Conversational Model Vinyals & Le (2015) They trained a seq2seq: Credit : Vinyals & Le ... with movie subtitles.

  9. Introduction Reasoning with Neural Networks Questions References A Neural Conversational Model Vinyals & Le (2015) Human : who is skywalker ? Machine : he is a hero . Human : what is the color of the sky ? Machine : blue . Human : how many legs does a cat have ? Machine : four , i think . Human : what is the purpose of living ? Machine : to live forever

  10. Introduction Reasoning with Neural Networks Questions References Research question Can we use neural networks for common sense reasoning?

  11. Introduction Reasoning with Neural Networks Questions References Long term goal

  12. Introduction Reasoning with Neural Networks Questions References Reasoning with Neural Networks Two main branches: - Common Sense embeddings. - Neural Reasoners.

  13. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Hinton (1990)

  14. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Hinton (1990)

  15. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings McClelland & Rogers (2003)

  16. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings McClelland & Rogers (2003)

  17. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings McClelland & Rogers (2003)

  18. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Socher et al. (2013) Reasoning with neural tensor networks for knowledge base completion.

  19. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Socher et al. (2013) Reasoning with neural tensor networks for knowledge base completion.

  20. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Socher et al. (2013)

  21. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) Recursive neural networks can learn logical semantics.

  22. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) Recursive neural networks can learn logical semantics.

  23. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) � − → � x ( l ) � � + − → → − y TreeRNN = f M b → − x ( r ) − → − → y TreeRNN + f ( − → x ( l ) T T [1 ...n ] − → x ( r ) ) y TreeRNTN =

  24. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  25. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  26. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  27. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  28. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  29. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  30. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  31. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  32. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  33. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) SICK textual entailment challenge

  34. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  35. Introduction Reasoning with Neural Networks Questions References Reasoning about facts

  36. Introduction Reasoning with Neural Networks Questions References Reasoning about facts The bAbI project (Weston et al. (2015)).

  37. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Three models have been proposed: - Dynamic Networks (Kumar et al. (2015)) - Memory Networks (Sukhbaatar et al. (2015)) - Neural Reasoner (Peng et al. (2015))

  38. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Sukhbaatar et al. (2015)

  39. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Kumar et al. (2015)

  40. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Peng et al. (2015)

  41. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Sukhbaatar et al. (2015)

  42. Introduction Reasoning with Neural Networks Questions References Reasoning about facts SLD resolution.

  43. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of?

  44. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of? A: cat (99.74%)

  45. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of? A: cat (99.74%) is emily afraid of gertrude?

  46. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of? A: cat (99.74%) is emily afraid of gertrude? A: cat (71.79%)

  47. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts the triangle is to the left of the red square the pink rectangle is below the triangle Questions is the red square to the right of the pink rectangle?

  48. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts the triangle is to the left of the red square the pink rectangle is below the triangle Questions is the red square to the right of the pink rectangle? A: yes (87%)

  49. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts the triangle is to the left of the red square the pink rectangle is below the triangle Questions is the red square to the right of the pink rectangle? A: yes (87%) is the red square to the left of the pink rectangle?

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Neural Module Networks for Reasoning Over Text Nitish Gupta , Kevin Lin , Dan Roth , Sameer Singh

Neural Module Networks for Reasoning Over Text Nitish Gupta , Kevin Lin , Dan Roth , Sameer Singh

Neural Module Networks for Reasoning Over Text Nitish Gupta , Kevin Lin , Dan Roth , Sameer Singh & Matt Gardner Presented by: Jigyasa Gupta Neural Modules Introduced in the paper Deep Compositional Question Answering with Neural

780 views • 54 slides
Evolution of Connections in SHRUTI Networks Joe Townsend, Ed Keedwell and Antony Galton

Evolution of Connections in SHRUTI Networks Joe Townsend, Ed Keedwell and Antony Galton

Evolution of Connections in SHRUTI Networks Joe Townsend, Ed Keedwell and Antony Galton Motivation Neural-Symbolic Reasoning: Adaptable representations of logic programs in neural networks. Can help us understand how reasoning might be

1.05k views • 40 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides
Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides
CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides
Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides
Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides
Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides
Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

Need for Machine . . . Neural Networks . . . How to Speed Up . . . Neural Networks: A . . . How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . . . Resulting Algorithm Faster by Taking Into How to

464 views • 17 slides
Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Paul Nuytten CPSC 607 Why Model Neural Networks? The Neuron The nervous

697 views • 4 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What well cover ... f ( x ) computer vision architectures - convolutional networks - data augmentation 1 - residual networks ... ...

1.29k views • 75 slides
NLO vs NNLO NLO calculations are/will be the workhorse of LHC physic. They are: Versatile

NLO vs NNLO NLO calculations are/will be the workhorse of LHC physic. They are: Versatile

Status of NNLO calculations Alexander Mitov Cavendish Laboratory NLO vs NNLO NLO calculations are/will be the workhorse of LHC physic. They are: Versatile Flexible Not always as accurate as we might want. NNLO, where possible,

315 views • 12 slides
Status of studies from HPAr TPC Alan Bross DUNE ND General meeting 25-Dec-2017 High-pressure Ar

Status of studies from HPAr TPC Alan Bross DUNE ND General meeting 25-Dec-2017 High-pressure Ar

Status of studies from HPAr TPC Alan Bross DUNE ND General meeting 25-Dec-2017 High-pressure Ar gas TPC (HPgTPC) status Much progress has been made in last 2 months on the design and performance simulation This is a work in progress,

732 views • 11 slides
CAPPS HR/PAYROLL MANAGER SELF-SERVICE TRAINING State Human Resources & Payroll Office Office

CAPPS HR/PAYROLL MANAGER SELF-SERVICE TRAINING State Human Resources & Payroll Office Office

CAPPS HR/PAYROLL MANAGER SELF-SERVICE TRAINING State Human Resources & Payroll Office Office of the Executive Director TEXAS MILITARY DEPARTMENT TEXAS MILITARY DEPARTMENT STRATEGY VISION: Americas premier state military comprised of

739 views • 48 slides
On Real-Time Status Updates over Symbol Erasure Channels Parimal Parag Austin Taghavi

On Real-Time Status Updates over Symbol Erasure Channels Parimal Parag Austin Taghavi

On Real-Time Status Updates over Symbol Erasure Channels Parimal Parag Austin Taghavi Jean-Francois Chamberland Electrical Communication Engineering Indian Institute of Science Electrical and Computer Engineering Texas A&M University

531 views • 14 slides
Kings College Hospital EHR: The Journey of a lifetime Richard Yorke, Head of EPR Large

Kings College Hospital EHR: The Journey of a lifetime Richard Yorke, Head of EPR Large

Kings College Hospital EHR: The Journey of a lifetime Richard Yorke, Head of EPR Large University General hospital in South East London Serving a population of 600,000 1600 inpatient beds 260,000 admissions 210, 000 ED

829 views • 43 slides
CarES Conference 5 th March 2014 Sharing the 500MW Target Local Energy Scotland CARES Conference

CarES Conference 5 th March 2014 Sharing the 500MW Target Local Energy Scotland CARES Conference

Local Energy Scotland CarES Conference 5 th March 2014 Sharing the 500MW Target Local Energy Scotland CARES Conference Sharing the 500MW Target Welcome Mike Thornton Director, Energy Saving Trust Scotland Local Energy Scotland CARES Conference

1.79k views • 141 slides
The Perpetual Game: Keeping Your Game Alive John Blickensderfer, President Chris Kleinfeld Ann

The Perpetual Game: Keeping Your Game Alive John Blickensderfer, President Chris Kleinfeld Ann

The Perpetual Game: Keeping Your Game Alive John Blickensderfer, President Chris Kleinfeld Ann Casstevens Guy Engineering Springfield Remanufacturing Corp Great Game of Business Tulsa, OK Springfield, MO Springfield, MO Getting to Know

289 views • 24 slides
CommStat 8/30/18 "The First Time I Felt It" by Stevie S. The first time I can remember

CommStat 8/30/18 "The First Time I Felt It" by Stevie S. The first time I can remember

CommStat 8/30/18 "The First Time I Felt It" by Stevie S. The first time I can remember ever drinking, at least more than just a single small sip for taste, given with "permission", I was the age of ten. I can remember going

1.17k views • 58 slides

More recommend