question answering on web data
play

Question Answering on Web Data Silei Xu CS294S April 9, 2020 Joint - PowerPoint PPT Presentation

Aug 26, 2023 •325 likes •1.23k views

Question Answering on Web Data Silei Xu CS294S April 9, 2020 Joint work with Giovanni Campagna, Sina Semnani, Jian Li, and Monica S. Lam Commercial Assistants Alexa User hand-codes question/code 1 by 1 get me an upscale restaurants What are

  1. Question Answering on Web Data Silei Xu CS294S April 9, 2020 Joint work with Giovanni Campagna, Sina Semnani, Jian Li, and Monica S. Lam

  2. Commercial Assistants Alexa User hand-codes question/code 1 by 1 get me an upscale restaurants What are the restaurants around here? What is the best restaurant? search for Chinese restaurants

  3. Commercial Assistants Alexa User hand-codes question/code 1 by 1 get me an upscale restaurants What are the restaurants around here? What is the best restaurant? search for Chinese restaurants 100K Alexa skills Sep 2019

  4. Commercial Assistants Alexa User hand-codes question/code 1 by 1 100K Alexa skills get me an upscale restaurants Sep 2019 What are the restaurants around here? What is the best restaurant? search for Chinese restaurants 1.8 billion websites

  5. Genie: Synthesize Question/Code from a Schema User Schema Name Price Cuisine … Property Annotations Genie 500 Domain- Independent Templates What is the <prop> of <subject>? What is the <subject>’s <prop>?

  6. Genie: Synthesize Question/Code from a Schema User get me an upscale restaurants Schema What are the restaurants around here? Name Price Cuisine … What is the best restaurant? search for Chinese restaurants What is the best restaurant within 10 miles? Find restaurants that serve Chinese or Japanese food Property Annotations What is the best non-Chinese restaurant near here? Show me a cheap restaurant with 5-star review. Genie Are there any restaurant with at least 4.5 stars? What is the phone number of Wendy’s? 500 Domain- I’m looking for an Italian fine dining restaurant. Independent Give me the best Italian restaurant. Templates Find me the best restaurant with 500 or more reviews Show me some restaurant with less than 10 reviews What is the <prop> of <subject>? What is the <subject>’s <prop>?

  7. The Web Has a Schema!

  8. The Web Has a Schema! • Schema.org • Structure data to mark up web pages • Mainly used by search engines • It covers many domains, including restaurants, hotels, people, recipes, products, news …

  9. The Web Has a Schema! • Schema.org <script type="application/ld+json"> • Structure data to mark up web pages { • Mainly used by search engines @type: "restaurant", name: "The French Laundry", • It covers many domains, including servesCuisine: “ French", restaurants, hotels, people, recipes, aggregateRating: { products, news … @type: "AggregateRating", reviewCount: 2527, ratingValue: 4.5 } ... Schema.org markup on Yelp }

  10. The Web Has a Schema! • Schema.org <script type="application/ld+json"> • Structure data to mark up web pages { • Mainly used by search engines @type: "restaurant", name: "The French Laundry", • It covers many domains, including servesCuisine: “ French", restaurants, hotels, people, recipes, aggregateRating: { products, news … @type: "AggregateRating", reviewCount: 2527, ratingValue: 4.5 } 40% of the websites use it! ... Schema.org markup on Yelp }

  11. Outline • Introduction to Schema.org • Represent Questions in ThingTalk • LUINet: NL to ThingTalk • Training data generation • Experimental results • Work in progress: automate everything!

  12. Introduction to Schema.org

  13. Graph Data Model of Schema.org

  14. Graph Data Model of Schema.org Organization legalName: Text slogan: Text aggregateRating: AggregateRating ...

  15. Graph Data Model of Schema.org class Organization legalName: Text slogan: Text aggregateRating: AggregateRating ...

  16. Graph Data Model of Schema.org class Organization properties legalName: Text slogan: Text aggregateRating: AggregateRating ...

  17. Graph Data Model of Schema.org class Organization types – primitive or class properties legalName: Text slogan: Text aggregateRating: AggregateRating ...

  18. Graph Data Model of Schema.org Organization AggregateRating legalName: Text ratingCount: Integer slogan: Text ratingValue: Integer aggregateRating: AggregateRating ... ...

  19. Schema.org Hierarchy Thing name: Text url: URL ... Organization ( Thing ) AggregateRating legalName: Text ratingCount: Integer slogan: Text ratingValue: Integer aggregateRating: AggregateRating ... ...

  20. Schema.org Hierarchy Thing name: Text url: URL ... Organization ( Thing ) AggregateRating legalName: Text ratingCount: Integer slogan: Text ratingValue: Integer aggregateRating: AggregateRating ... ... LocalBusiness ( Place , Organization ) openingHours: Text priceRange: Text ...

  21. Some useful tools • Google Structured Data Testing Tool • Show schema.org markups in a web page • Google Custom Search • Search for pages that contain certain schema.org domains

  22. ThingTalk for Questions

  23. ThingTalk for QA

  24. ThingTalk for QA @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => => notify Show me restaurants in Stanford

  25. ThingTalk for QA @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” => notify Show me Chinese restaurants in Stanford

  26. ThingTalk for QA @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” => notify Show me Chinese restaurants in Stanford

  27. ThingTalk for QA sort aggregateRating.ratingValue desc of ( @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” ) => notify Show me top-rated Chinese restaurants in Stanford

  28. ThingTalk for QA sort aggregateRating.ratingValue desc of ( @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” ) => notify Show me top-rated Chinese restaurants in Stanford

  29. ThingTalk for QA sort aggregateRating.ratingValue desc of ( @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” ) => notify join ( @QA.Review(), in_array(id, review) && author = “ bob ” ) Show me top-rated Chinese restaurants in Stanford reviewed by Bob

  30. ThingTalk for QA

  31. ThingTalk for QA

  32. ThingTalk for QA …

  33. LUINet: NL to ThingTalk

  34. Natural Language Programming LUINet Natural language ThingTalk What is the top-rated Chinese sort aggregateRating.ratingValue desc of ( @ QA.restaurant(), restaurant in Palo Alto? geo == new MakeLocation (“ Stanford ”) && servesCuisine =~ “ Chinese ” )

  35. Genie Pipeline LUINet Natural language ThingTalk

  36. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant LUINet Natural language ThingTalk

  37. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Domain-independent Templates What is the <prop> of <table>? What is the <table >’s <prop>? LUINet Natural language ThingTalk

  38. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates What is the <prop> of <table>? What is the <table >’s <prop>? LUINet Natural language ThingTalk

  39. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates Paraphrase What is the <prop> of <table>? What is the <table >’s <prop>? LUINet Natural language ThingTalk

  40. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates Paraphrase What is the <prop> of <table>? What is the <table >’s <prop>? Parameter & data augmentation LUINet Natural language ThingTalk

  41. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates Paraphrase What is the <prop> of <table>? What is the <table >’s <prop>? Parameter & data augmentation Training Data LUINet Natural language ThingTalk

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Question Answering What is Ques+on Answering? Dan Jurafsky Ques%on

Question Answering What is Ques+on Answering? Dan Jurafsky Ques%on

Question Answering What is Ques+on Answering? Dan Jurafsky Ques%on Answering One of the oldest NLP tasks (punched card systems in 1961) Simmons, Klein,

856 views • 73 slides
A Multilingual Hybrid Question-Answering System Cross-Lingual Open-Domain Question Answering

A Multilingual Hybrid Question-Answering System Cross-Lingual Open-Domain Question Answering

A Multilingual Hybrid Question-Answering System Cross-Lingual Open-Domain Question Answering Gnter Neumann, Bogdan Sacaleanu 30th DFKI SAB MEETING 04/04/2006 German Research Center for Artificial Intelligence Inference Linguistic World

428 views • 20 slides
Semantic Question Answering on Big Data Tatiana Erekhinskaya July, 2016 The Goal Challenge:

Semantic Question Answering on Big Data Tatiana Erekhinskaya July, 2016 The Goal Challenge:

Semantic Question Answering on Big Data Tatiana Erekhinskaya July, 2016 The Goal Challenge: Find answers to complex questions in large structured and unstructured data resources Sample question: List Chinese researchers who

240 views • 20 slides
Question Answering and AnswerFinder Diego Moll a Centre for Language Technology Department of

Question Answering and AnswerFinder Diego Moll a Centre for Language Technology Department of

AnswerFinder Stages in Question Answering Learning Question Answering Rules Question Answering and AnswerFinder Diego Moll a Centre for Language Technology Department of Computer Science Macquarie University Athens, 4 July 2007 Diego

1.03k views • 64 slides
CS345a Data Mining Project A Web Based Question Answering System Vincenzo Di Nicola Jyotika

CS345a Data Mining Project A Web Based Question Answering System Vincenzo Di Nicola Jyotika

CS345a Data Mining Project A Web Based Question Answering System Vincenzo Di Nicola Jyotika Prasad The Ultimate question answering system What is the meaning of life? Who are we? Why are we doing CS? Or, less philosophically,

162 views • 13 slides
Statistical NLP Spring 2011 Lecture 26: Question Answering Dan Klein UC Berkeley Question

Statistical NLP Spring 2011 Lecture 26: Question Answering Dan Klein UC Berkeley Question

Statistical NLP Spring 2011 Lecture 26: Question Answering Dan Klein UC Berkeley Question Answering Following largely from Chris Mannings slides, which includes slides originally borrowed from Sanda Harabagiu, ISI, Nicholas

414 views • 20 slides
Question Answering Statistical NLP Following largely from Chris Mannings slides, which

Question Answering Statistical NLP Following largely from Chris Mannings slides, which

Question Answering Statistical NLP Following largely from Chris Mannings slides, which includes Spring 2011 slides originally borrowed from Sanda Harabagiu, ISI, Nicholas Kushmerick. Lecture 26: Question Answering Dan Klein UC

531 views • 7 slides
Designing deep architectures for Visual Question Answering Matthieu Cord Sorbonne University

Designing deep architectures for Visual Question Answering Matthieu Cord Sorbonne University

Designing deep architectures for Visual Question Answering Matthieu Cord Sorbonne University valeo.ai research lab. Paris Thanks to H. Ben-younes, R. Cadne Visual Question Answering Question Answering: + What does Claudia do? Visual

831 views • 68 slides
Question Answering and Reading Comprehension Kevin Duh Fall 2019, Intro to HLT, Johns Hopkins

Question Answering and Reading Comprehension Kevin Duh Fall 2019, Intro to HLT, Johns Hopkins

Question Answering and Reading Comprehension Kevin Duh Fall 2019, Intro to HLT, Johns Hopkins University What is Question Answering? Its a field concerned with building systems that answer questions posed in natural language Question

484 views • 36 slides
Towards End-to-End Reasoning for Question Answering Minjoon Seo Department of Computer Science

Towards End-to-End Reasoning for Question Answering Minjoon Seo Department of Computer Science

Towards End-to-End Reasoning for Question Answering Minjoon Seo Department of Computer Science &amp; Engineering University of Washington September 29, 2016 @ Samsung AI Lab What is reasoning? Simple Question Answering Model What is

727 views • 60 slides
Question-Answering: Shallow &amp; Deep Techniques for NLP Deep Processing Techniques for NLP

Question-Answering: Shallow &amp; Deep Techniques for NLP Deep Processing Techniques for NLP

Question-Answering: Shallow &amp; Deep Techniques for NLP Deep Processing Techniques for NLP Ling 571 March 12, 2014 (Examples from Dan Jurafsky) Roadmap Question-Answering: Definitions &amp; Motivation Basic pipeline:

426 views • 32 slides
Factoid Question Answering Roy Aslan (ra2752@Columbia.edu) A Neural Network for Factoid

Factoid Question Answering Roy Aslan (ra2752@Columbia.edu) A Neural Network for Factoid

Factoid Question Answering Roy Aslan (ra2752@Columbia.edu) A Neural Network for Factoid Question Answering over Paragraphs Mohit Iyyer, Jordan Boyd-Graber, Leonardo Claudino, Richard Socher, and Hal Daum III Task and Setting Factoid

504 views • 21 slides
Question-Answering: Shallow &amp; Deep Techniques for NLP Ling571 Deep Processing Techniques

Question-Answering: Shallow &amp; Deep Techniques for NLP Ling571 Deep Processing Techniques

Question-Answering: Shallow &amp; Deep Techniques for NLP Ling571 Deep Processing Techniques for NLP March 9, 2011 Examples from Dan Jurafsky) Roadmap Question-Answering: Definitions &amp; Motivation Basic pipeline:

993 views • 84 slides
Phrase-Indexed Question Answering : A New Challenge for Scalable Document Comprehension Minjoon

Phrase-Indexed Question Answering : A New Challenge for Scalable Document Comprehension Minjoon

Phrase-Indexed Question Answering : A New Challenge for Scalable Document Comprehension Minjoon Seo, Tom Kwiatkowski, Ankur Parikh, Ali Farhadi, Hannaneh Hajishirzi Question Answering? 1961 Model Barack Obama (1961-present) was the 44 th

460 views • 32 slides
Question Classification in English-Chinese Cross-Language Question Answering: An Integrated

Question Classification in English-Chinese Cross-Language Question Answering: An Integrated

Question Classification in English-Chinese Cross-Language Question Answering: An Integrated Genetic Algorithm and Machine Learning Approach Min-Yuh Day 1, 2 , Chorng-Shyong Ong 2 , and Wen-Lian Hsu 1,* , Fellow, IEEE 1 Institute of Information

608 views • 21 slides
Self-Critical Reasoning for Robust Visual Question Answering Jialin Wu and Raymond J. Mooney

Self-Critical Reasoning for Robust Visual Question Answering Jialin Wu and Raymond J. Mooney

Self-Critical Reasoning for Robust Visual Question Answering Jialin Wu and Raymond J. Mooney Visual Question Answering (VQA) Common VQA system What utensil is pictured? Knife (0.72) Answer Prediction Fork Visual feature set

513 views • 11 slides
Introduction to Computational Topology Ahmed Abdelkader Guest Lecture CMSC 754 Spring 2020

Introduction to Computational Topology Ahmed Abdelkader Guest Lecture CMSC 754 Spring 2020

Introduction to Computational Topology Ahmed Abdelkader Guest Lecture CMSC 754 Spring 2020 May 7th, 2020 Ahmed Abdelkader (CS@UMD) Introduction to Computational Topology May 7th, 2020 1 / 41 Introduction Pages 13 Early Topological

1.93k views • 147 slides
Multigrid methods for zero-sum two player stochastic games with mean reward Sylvie Detournay and

Multigrid methods for zero-sum two player stochastic games with mean reward Sylvie Detournay and

Multigrid methods for zero-sum two player stochastic games with mean reward Sylvie Detournay and Marianne Akian INRIA Saclay and CMAP, Ecole Polytechnique (France) 15th Copper Mountain Conference on Multigrid Methods 27 March - 1 April,

299 views • 28 slides
Recent Advances in Two-loop Superstrings Eric DHoker Institut des Hautes Etudes Scientifiques,

Recent Advances in Two-loop Superstrings Eric DHoker Institut des Hautes Etudes Scientifiques,

Recent Advances in Two-loop Superstrings Eric DHoker Institut des Hautes Etudes Scientifiques, 2014 May 6 Eric DHoker Recent Advances in Two-loop Superstrings Outline 1. Overview of two-loop superstring methods, including global issues;

627 views • 31 slides
Why You Should Run TPC-DS: A Workload Analysis Meikel Poess Raghunath Othayoth Nambiar Dave

Why You Should Run TPC-DS: A Workload Analysis Meikel Poess Raghunath Othayoth Nambiar Dave

Why You Should Run TPC-DS: A Workload Analysis Meikel Poess Raghunath Othayoth Nambiar Dave Walrath Sybase Inc Oracle USA Hewlett-Packard Company (in absentia) Agenda Transaction Processing Performance Council (TPC) Scope of TPC-DS

679 views • 24 slides
The Extended SPaRKy Restaurant Corpus designing a corpus with variable information density David

The Extended SPaRKy Restaurant Corpus designing a corpus with variable information density David

The Extended SPaRKy Restaurant Corpus designing a corpus with variable information density David M. Howcroft Dietrich Klakow Vera Demberg Department of Language Science and Technology Saarland Informatics Campus, Saarland University, Germany

759 views • 54 slides
Inferring Restaurant Styles by Mining Crowd Sourced Photos from User-Review Websites Haofu Liao,

Inferring Restaurant Styles by Mining Crowd Sourced Photos from User-Review Websites Haofu Liao,

@IEEE BigData 2016 Inferring Restaurant Styles by Mining Crowd Sourced Photos from User-Review Websites Haofu Liao, Yuncheng Li, Tianran Hu and Jiebo Luo Department of Computer Science UNIVERSITY of ROCHESTER Part I - The Problem UNIVERSITY

617 views • 25 slides
Noun-Verb Decomposition Nouns Verbs Restaurant [Regular, Catering, Take- has (information) Out]

Noun-Verb Decomposition Nouns Verbs Restaurant [Regular, Catering, Take- has (information) Out]

Noun-Verb Decomposition Nouns Verbs Restaurant [Regular, Catering, Take- has (information) Out] (Location, Type of food, Hours of operation, Reservations) Reservation Information Location Hours of operation Things SWEN-261 Account Introduction

598 views • 22 slides
Systems programming Thread management (Cont.) Synchronization using Semaphores Most of the

Systems programming Thread management (Cont.) Synchronization using Semaphores Most of the

Systems programming Thread management (Cont.) Synchronization using Semaphores Most of the slides in this lecture are either from or adapted from the slides provided by Dr. Ahmad Barghash Semaphores Introduced in 1965 by E. Dijkstra It

727 views • 34 slides

More recommend