a neural conversation model
play

A Neural Conversation Model Aadil Hayat (13002) Masare Akshay Sunil - PowerPoint PPT Presentation

Oct 09, 2022 •132 likes •249 views

A Neural Conversation Model Aadil Hayat (13002) Masare Akshay Sunil (12403) Problem Statement To create a chatter bot capable to chat in natural language as humans do. DNNs in Conversations The inputs and targets are required to be of

  1. A Neural Conversation Model Aadil Hayat (13002) Masare Akshay Sunil (12403)

  2. Problem Statement • To create a chatter bot capable to chat in natural language as humans do.

  3. DNNs in Conversations • The inputs and targets are required to be of fixed dimensionality • Many problems that deal with sequence of inputs or targets where the dimensionality can not be predicted a-priori • Eg. Speech Recognition, Machine Translation and Question-Answering

  4. Seq2Seq • One LSTM reads a whole Sequence at a time to generate a single vector with a large dimensionality. • The Second LSTM is a RNN which is conditioned on the input sequence.

  5. A Conversational Model • A Translation Model using Seq2Seq will be easier than a conversational model due to no context required. • Input sequence is the concatenation of what has been conversed so far. • Lack of general world knowledge is another limitation of the purely unsupervised model.

  6. IT Helpdesk Troubleshooting VPN Issue

  7. The Dataset • OpenSubtitles dataset Tiedemann 2009. • Movie conversations in XML format. • Training Dataset : 62M sentences (923M tokens) • Validation Dataset : 26M sentences (395 tokens) • Quite large but noisy • This is an open-domain conversation dataset so expected results are quite fascinating.

  8. Cleverbot vs NCM Cleverbot Neural Conversational Model

  9. References [1] Weizenbaum, J. ELIZA - A Computer Program For the Study of Natural Language Communication Between Man And Machine. Communications of the ACM, Vol 9, 1966. [2] Huang, J., Zhou, M., Yang, D. Extracting Chatbot Knowledge from Online Discussion Forums. IJCAI07-066, 2007. [3] Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Gao, J., Dolan, B., and Nie, J.-Y. A neural network approach to context-sensitive generation of conversational responses. In Proceedings of NAACL, 2015. [4] Sutskever, I., Vinyals, O., and Le, Q. V. Sequence to sequence learning with neural networks. In NIPS, 2014.

  10. References [5] Hochreiter, S. and Schmidhuber, J. Long short-term memory . Neural Computation, 1997. [6] Vinyals, O., Le, Q. V. A Neural Conversational Model. arXiv:1506.05869v3, 2015. [7] Shang, L., Lu, Z., and Li, H. Neural responding machine for short-text conversation. In Proceedings of ACL, 2015. [8] Tiedemann, J. News from OPUS - A collection of multilingual parallel corpora with tools and interfaces. In Nicolov, N., Bontcheva, K., Angelova, G., and Mitkov, R. (eds.), Recent Advances in Natural Language Processing, volume V, pp. 237 – 248. John Benjamins, Amsterdam/Philadelphia, Borovets, Bulgaria, 2009. ISBN 978 90 272 4825 1.

  11. Questions?

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural

8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural

8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural machine translation, which uses an encoder to encode sentences as a fixed-length vector. However, in some ways, this view is overly simplified, and

635 views • 9 slides
Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Paul Nuytten CPSC 607 Why Model Neural Networks? The Neuron The nervous

697 views • 4 slides
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation EMNLP19 Deepanway Ghosal, Navonil Majumder, Soujanya Poria , Niyati Chhaya and Alexander Gelbukh Singapore University of Technology and Design, Singapore

995 views • 21 slides
Neural Response Ranking for Social Conversation: A Data-Efficient Approach Igor Shalyminov, Ond

Neural Response Ranking for Social Conversation: A Data-Efficient Approach Igor Shalyminov, Ond

Neural Response Ranking for Social Conversation: A Data-Efficient Approach Igor Shalyminov, Ond ej Duek, and Oliver Lemon School of Mathematical and Computer Sciences Heriot-Watt University 31 October 2018 Outline Introduction. Amazon

522 views • 13 slides
Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A

Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A

Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Why Model Neural Networks? The nervous system is a

1.22k views • 23 slides
Grounding Neural Conversation Models into the Real World Michel Galley SCAI October 1 st , 2017

Grounding Neural Conversation Models into the Real World Michel Galley SCAI October 1 st , 2017

Grounding Neural Conversation Models into the Real World Michel Galley SCAI October 1 st , 2017 Inform ormation ation Retri trieval eval Conver nversati sation onal l AI AI Natu Na tura ral l La Languag guage Dialogu ogue Pr

742 views • 30 slides
Exemplar Encoder Decoder for Neural Conversation Generation By Gaurav Pandey, Danish

Exemplar Encoder Decoder for Neural Conversation Generation By Gaurav Pandey, Danish

Exemplar Encoder Decoder for Neural Conversation Generation By Gaurav Pandey, Danish Contractor, Vineet Kumar and Sachindra Joshi IBM Research AI Generative Models for Conversations Context Context Context Response Decoder Embedding

437 views • 12 slides
1 ! Knowing The Right Conversation What would the right conversation look and sound like

1 ! Knowing The Right Conversation What would the right conversation look and sound like

1 ! Knowing The Right Conversation What would the right conversation look and sound like when you initially engage others? If you listened to your colleagues conversation when she or he was dealing with someone what would it reveal?

1.05k views • 70 slides
1 Real Neural Learning Artificial Neuron Model Model network as a graph with cells as nodes

1 Real Neural Learning Artificial Neuron Model Model network as a graph with cells as nodes

Neural Networks Analogy to biological neural systems, the most robust learning systems we know. Attempt to understand natural biological systems CS 391L: Machine Learning through computational modeling. Massive parallelism allows

92 views • 6 slides
Stimulus-Based Conversation Conquering SBC using the SEP model An example of an SBC question a)

Stimulus-Based Conversation Conquering SBC using the SEP model An example of an SBC question a)

Stimulus-Based Conversation Conquering SBC using the SEP model An example of an SBC question a) Look at the road sign. Who do you think it is put up for? Why do you think so? Do you think it is important for drivers to obey the warning on

309 views • 27 slides
Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron

Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron

CS4501: Introduction to Computer Vision Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron model Neural Networks multilayer perceptron model (MLP) Backpropagation Todays Class

789 views • 34 slides
MeanSum : A Neural Model for Unsupervised Multi-Document Abstractive Summarization Eric Chu *

MeanSum : A Neural Model for Unsupervised Multi-Document Abstractive Summarization Eric Chu *

MeanSum : A Neural Model for Unsupervised Multi-Document Abstractive Summarization Eric Chu * Peter J. Liu * MIT Media Lab Google Brain ICML 2019 Text summarization overview Extractive Abstractive Non-neural Human Supervised Neural

317 views • 14 slides
LSTM Neural Reordering Model for Statistical Machine Translation Yiming Cui, Shijin Wang,

LSTM Neural Reordering Model for Statistical Machine Translation Yiming Cui, Shijin Wang,

LSTM Neural Reordering Model for Statistical Machine Translation Yiming Cui, Shijin Wang, Jianfeng Li iFLYTEK Research June 14, 2016 O UTLINE Lexicalized Reordering Model LSTM Neural Reordering Model Experiments & Analyses

423 views • 20 slides
A Generative Model of Group Conversation Hannah Morrison Chris Martens About Me Junior at

A Generative Model of Group Conversation Hannah Morrison Chris Martens About Me Junior at

A Generative Model of Group Conversation Hannah Morrison Chris Martens About Me Junior at North Carolina State University Favorite games: Transistor , Hellblade: Senuas Sacrifice , Life is Strange , Overwatch

546 views • 27 slides
Recurrent Language Models CMSC 470 Marine Carpuat Toward a Neural Language Model Figures by

Recurrent Language Models CMSC 470 Marine Carpuat Toward a Neural Language Model Figures by

Recurrent Language Models CMSC 470 Marine Carpuat Toward a Neural Language Model Figures by Philipp Koehn (JHU) Count-based n-gram models vs. feedforward neural networks Pros of feedforward neural LM Word embeddings capture

547 views • 16 slides
Recurrent Language Models CMSC 470 Marine Carpuat Toward a Neural Language Model Figures by

Recurrent Language Models CMSC 470 Marine Carpuat Toward a Neural Language Model Figures by

Recurrent Language Models CMSC 470 Marine Carpuat Toward a Neural Language Model Figures by Philipp Koehn (JHU) Count-based n-gram models vs. feedforward neural networks Pros of feedforward neural LM Word embeddings capture

279 views • 15 slides
1/37 Lesson: How I Learned to Stop Worrying and Love the Bot 2/37 Lesson: How I Learned to Stop

1/37 Lesson: How I Learned to Stop Worrying and Love the Bot 2/37 Lesson: How I Learned to Stop

1/37 Lesson: How I Learned to Stop Worrying and Love the Bot 2/37 Lesson: How I Learned to Stop Worrying and Love the Bot 3/37 Lesson: How I Learned to Stop Worrying and Love the Bot Poll 4/37 Lesson: How I Learned to Stop Worrying and Love

1.15k views • 37 slides
Computational Linguistics The Future of NLP We can study anything about language ... 1.

Computational Linguistics The Future of NLP We can study anything about language ... 1.

Computational Linguistics The Future of NLP We can study anything about language ... 1. Formalize some insights 2. Study the formalism mathematically A Few Random Remarks 3. Develop & implement algorithms 4. Test on real

362 views • 4 slides
The ShanghAI Lectures An experiment in global teaching Fabio Bonsignorio The BioRobotics

The ShanghAI Lectures An experiment in global teaching Fabio Bonsignorio The BioRobotics

The ShanghAI Lectures An experiment in global teaching Fabio Bonsignorio The BioRobotics Institute, SSSA and Heron Robots Today from the BioRobotics Institute, Pontedera (PI)

651 views • 45 slides
Language and Computers Speech acts Rules Early dialogue Dialog Systems systems ELIZA Other

Language and Computers Speech acts Rules Early dialogue Dialog Systems systems ELIZA Other

Language and Computers Dialog Systems Introduction Why dialogue? Human dialogue Basic facts Dialogue moves Language and Computers Speech acts Rules Early dialogue Dialog Systems systems ELIZA Other chatterbots Modern dialogue Based

778 views • 38 slides
1 User frustration Error messages The application Word Wonder has unexpectedly quit due to a

1 User frustration Error messages The application Word Wonder has unexpectedly quit due to a

Overview Expressive interfaces Understanding how how the appearance of an interface can elicit positive responses interfaces affect users Negative aspects how computers frustrate users Anthropomorphism and

402 views • 5 slides
Ethernet Daniel Morat Area de Ingeniera Telemtica Departamento de Automtica y

Ethernet Daniel Morat Area de Ingeniera Telemtica Departamento de Automtica y

Ethernet Daniel Morat Area de Ingeniera Telemtica Departamento de Automtica y Computacin Universidad Pblica de Navarra daniel.morato@unavarra.es http://www.tlm.unavarra.es/asignaturas/lpr LANs Ethernet Nos centramos ahora en

403 views • 17 slides
61A Lecture 6 Announcements Recursive Functions Recursive Functions 4 Recursive Functions

61A Lecture 6 Announcements Recursive Functions Recursive Functions 4 Recursive Functions

61A Lecture 6 Announcements Recursive Functions Recursive Functions 4 Recursive Functions Definition: A function is called recursive if the body of that function calls itself, either directly or indirectly 4 Recursive Functions Definition:

1.08k views • 88 slides
Commentaries on Problems JUDGE TEAM ACM ICPC 2016 ASIA TSUKUBA REGIONAL Problem Set Design

Commentaries on Problems JUDGE TEAM ACM ICPC 2016 ASIA TSUKUBA REGIONAL Problem Set Design

Commentaries on Problems JUDGE TEAM ACM ICPC 2016 ASIA TSUKUBA REGIONAL Problem Set Design Objectives All the teams should be able to solve at least one problem All the problems should be solved by at least one team No team should

806 views • 69 slides

More recommend