(Question Answering and Dialogue Systems) 063) - PowerPoint PPT Presentation

Tamkang University ��������� (Question Answering and Dialogue Systems) 0�63��������)��� ���� ������������� �� /5123���F����H��� ��������)(� �-��� ��9:����P G� �H����GG� Min-Yuh Day ��� Associate Professor ��� Dept. of Information Management, Tamkang University ���� ������ http://mail. tku.edu.tw/myday/ 1 2020-06-19

Topics 1. ��������������� (Core Technologies of Natural Language Processing and Text Mining) 2. ������������� (Artificial Intelligence for Text Analytics: Foundations and Applications) 3. �������� (Feature Engineering for Text Representation) 4. ����������� (Semantic Analysis and Named Entity Recognition; NER) 5. ������������� (Deep Learning and Universal Sentence-Embedding Models) 6. ��������� (Question Answering and Dialogue Systems) 2

Question Answering and Dialogue Systems 3

Outline • Question Answering • Dialogue Systems 4

IMTKU System Architecture for NTCIR-13 QALab-3 Question (XML) JA&EN Complex Essay Translator Question Analysis Simple Essay Stanford True-or-False CoreNLP Factoid Document Retrieval Wikipedia Slot-Filling Unique Answer Extraction Word Embedding Answer Generation Wiki Word2Vec Answer (XML) 5 NTCIR-13 Conference, December 5-8, 2017, Tokyo, Japan

System Architecture of Intelligent Dialogue and Question Answering System User Question Input Deep Learning RNN TensorFlow Dialogue Intention LSTM Question Analysis Detection Python GRU NLTK AIML Dialogue Document Retrieval Dialogue KB AIML KB Engine IR Answer Extraction Real Time Cloud Dialogue Resource API Answer Answer Deep Learning Validation Generation System Response Answer Generator 6

IMTKU Emotional Dialogue System Architecture 1 3 4 Retrieval-Based Model Emotion Response Classification Ranking Model Generation- Based Model 2 7 NTCIR-14 Conference, June 10-13, 2019, Tokyo, Japan

The system architecture of IMTKU retrieval-based model for NTCIR-14 STC-3 Retrieval-Based Model 1 Post Corpus Word Segmentation Building Index Retrieval Model Keyword Distinct Word2Vec Solr Emotion Boolean Result Similarity Matching Matching Query Data Ranking Emotion Retrieval- Classification Based Response 8 NTCIR-14 Conference, June 10-13, 2019, Tokyo, Japan

The system architecture of IMTKU generation-based model for NTCIR-14 STC-3 Generation-Based Model 2 Post Training Data Word Segmentation Building Word Index Short Text Word Emotion Classifier Embedding Training Data Trained Model Seq2seq model Generative Model Emotion Matching Generation-Based Word2Vec Similarity Response Ranking NTCIR-14 Conference, June 10-13, 2019, Tokyo, Japan 9

The system architecture of IMTKU emotion classification model for NTCIR-14 STC-3 Emotion Classification Model 3 MLP Training LSTM Dataset BiLSTM Corpus Emotion Classification Emotion Emotion Testing Classification Prediction Dataset Model 10 NTCIR-14 Conference, June 10-13, 2019, Tokyo, Japan

The system architecture of IMTKU Response Ranking for NTCIR-14 STC-3 Response Ranking 4 Chinese STC3 Stop Vector of Segmentation 1.2 million data Words Corpus Corpus Word2Vec using (300 dimensions) Removal Jieba 11 NTCIR-14 Conference, June 10-13, 2019, Tokyo, Japan

Short Text Conversation Task (STC-3) Chinese Emotional Conversation Generation (CECG) Subtask Source: http://coai.cs.tsinghua.edu.cn/hml/challenge.html 12

NTCIR Short Text Conversation STC-1, STC-2, STC-3 Source: https://waseda.app.box.com/v/STC3atNTCIR-14 13

Chatbots: Evolution of UI/UX 14 Source: https://bbvaopen4u.com/en/actualidad/want-know-how-build-conversational-chatbot-here-are-some-tools

AI Dialogue System 15

Dialogue Subtasks Task-Oriented Dialogue Dialogue Generation Systems Short-Text Conversation 16 Source: https://paperswithcode.com/area/natural-language-processing/dialogue

Chatbot Dialogue System Intelligent Agent 17

Chatbot 18 Source: https://www.mdsdecoded.com/blog/the-rise-of-chatbots/

Dialogue System Source: Serban, I. V., Lowe, R., Charlin, L., & Pineau, J. (2015). A survey of available corpora for building data-driven dialogue systems. arXiv 19 preprint arXiv:1512.05742 .

Overall Architecture of Intelligent Chatbot 20 Source: Borah, Bhriguraj, Dhrubajyoti Pathak, Priyankoo Sarmah, Bidisha Som, and Sukumar Nandi. "Survey of Textbased Chatbot in Perspective of Recent Technologies." In International Conference on Computational Intelligence, Communications, and Business Analytics, pp. 84-96. Springer, Singapore, 2018.

Can machines think? (Alan Turing ,1950) Source: Cahn, Jack. "CHATBOT: Architecture, Design, & Development." 21 PhD diss., University of Pennsylvania, 2017.

Chatbot “online human-computer dialog system with natural language.” Source: Cahn, Jack. "CHATBOT: Architecture, Design, & Development." 22 PhD diss., University of Pennsylvania, 2017.

Chatbot Conversation Framework 23 Source: https://chatbotslife.com/ultimate-guide-to-leveraging-nlp-machine-learning-for-you-chatbot-531ff2dd870c

Chatbots Bot Maturity Model Customers want to have simpler means to interact with businesses and get faster response to a question or complaint. 24 Source: https://www.capgemini.com/2017/04/how-can-chatbots-meet-expectations-introducing-the-bot-maturity/

From E-Commerce to Conversational Commerce: Chatbots and Virtual Assistants 25 Source: http://www.guided-selling.org/from-e-commerce-to-conversational-commerce/

Conversational Commerce: eBay AI Chatbots 26 Source: https://www.forbes.com/sites/rachelarthur/2017/07/19/conversational-commerce-ebay-ai-chatbot/

Hotel Chatbot Intent Detection Slot Filling 27 Source: https://sdtimes.com/amazon/guest-view-capitalize-amazon-lex-available-general-public/

H&M’s Chatbot on Kik 28 Source: http://www.guided-selling.org/from-e-commerce-to-conversational-commerce/

Uber’s Chatbot on Facebook’s Messenger Uber’s chatbot on Facebook’s messenger - one main benefit: it loads much faster than the Uber app 29 Source: http://www.guided-selling.org/from-e-commerce-to-conversational-commerce/

Savings Bot 30 Source: https://chatbotsmagazine.com/artificial-intelligence-ai-and-fintech-part-1-7cae1e67dc13

Mastercard Makes Commerce More Conversational 31 Source: https://newsroom.mastercard.com/press-releases/mastercard-makes-commerce-more-conversational-with-launch-of-chatbots-for-banks-and-merchants/

Bot Life Cycle and Platform Ecosystem 32

The Bot Lifecycle 33 Source: https://chatbotsmagazine.com/the-bot-lifecycle-1ff357430db7

34 Source: https://www.oreilly.com/ideas/infographic-the-bot-platform-ecosystem

35 Source: https://www.oreilly.com/ideas/infographic-the-bot-platform-ecosystem

36 Source: https://venturebeat.com/2016/08/11/introducing-the-bots-landscape-170-companies-4-billion-in-funding-thousands-of-bots/

37 Source: https://medium.com/@RecastAI/2017-messenger-bot-landscape-a-public-spreadsheet-gathering-1000-messenger-bots-f017fdb1448a /

How to Build Chatbots 38 Source: Igor Bobriakov (2018), https://activewizards.com/blog/a-comparative-analysis-of-chatbots-apis/

Chatbot Frameworks and AI Services • Bot Frameworks – Botkit – Microsoft Bot Framework – Rasa NLU • AI Services – Wit.ai – api.ai – LUIS.ai – IBM Watson 39 Source: Igor Bobriakov (2018), https://activewizards.com/blog/a-comparative-analysis-of-chatbots-apis/

Chatbot Frameworks 40 Source: Igor Bobriakov (2018), https://activewizards.com/blog/a-comparative-analysis-of-chatbots-apis/

41 Source: Igor Bobriakov (2018), https://activewizards.com/blog/a-comparative-analysis-of-chatbots-apis/

Transformer (Attention is All You Need) (Vaswani et al., 2017) Source: Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 42 "Attention is all you need." In Advances in neural information processing systems , pp. 5998-6008. 2017.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding BERT (Bidirectional Encoder Representations from Transformers) Overall pre-training and fine-tuning procedures for BERT Source: Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova (2018). 43 "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint arXiv:1810.04805.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding BERT (Bidirectional Encoder Representations from Transformers) BERT input representation Source: Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova (2018). 44 "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint arXiv:1810.04805.