Commonsense Properties from Query Logs and Question Answering - PowerPoint PPT Presentation

Commonsense Properties from Query Logs and Question Answering Forums Julien Romero, Simon Razniewski, Koninika Pal, Jeff Z. Pan, Archit Sakhadeo, Gerhard Weikum Une école de l’IMT

Goal ■ Mine Commonsense Knowledge (CSK) about : Object properties − Human behavior − General concepts − ■ Focus on salient properties ■ Examples : (bananas, are, edible) − (children, like, bananas) − ■ Applications : Chatbot, Question Answering, Visual content understanding, Search engine queries interpretation, ... 2 Une école de l’IMT QUASIMODO 2019/11/05

Challenges ■ Sparseness and bias ■ Rarely expressed ■ Non-encyclopedic (no Wikipedia) ■ Noise and high bias on online content 3 Une école de l’IMT QUASIMODO 2019/11/05

Previous Work ■ Traditional Knowledge Bases No commonsense − ■ ConceptNet Manual, does not scale − ■ Webchild Focus on possible properties, not salient ones − ■ TupleKB Domain specific − 4 Une école de l’IMT QUASIMODO 2019/11/05

General Pipeline 5 Une école de l’IMT QUASIMODO 2019/11/05

Candidate Gathering ■ Main idea : Extract facts from questions When asking a question, make assumptions about the world − Why are bananas yellow? Bananas are yellow! Harvest human curiosity, « wisdom of the crowds » − 6 Une école de l’IMT QUASIMODO 2019/11/05

Candidate Gathering – Query Logs ■ Indirect access to the query logs through autocompletion 7 Une école de l’IMT QUASIMODO 2019/11/05

Candidate Gathering – QA Forums Quora Yahoo! Answers (semi-manually) (research datasets) why-how questions Answers.com Reddit (sitemap) (dump) 8 Une école de l’IMT QUASIMODO 2019/11/05

Candidate Gathering – Statistics 9 Une école de l’IMT QUASIMODO 2019/11/05

Candidate Gathering – Results ■ Questions transformed to statements then to triples using OpenIE techniques Q2S Why do lions often hunt zebras? Lions often hunt zebras OpenIE (lions, often eat, zebras) Modality (lions, eat, zebras, often) Positivity (lions, eat, zebras, often, positive) Source (lions, eat, zebras, often, positive, Google, 0.4) 10 Une école de l’IMT QUASIMODO 2019/11/05

Corroboration ■ Reduce noise thanks to additional signals from : Wikipedia and Simple Wikipedia − Answer snippets from search engines − Google Books − Image Tags from OpenImages and Flickr − Google’s Conceptual Captions dataset − ■ Train Naive Bayes from all signals from 700 manually annotated triples (TuplesKB requires 70.000) Precision of 61% − 11 Une école de l’IMT QUASIMODO 2019/11/05

Ranking + TODO Example ■ From Corroboration, get plausibility score π ■ Define a probability from it: ■ Derive a typicality τ and a saliency σ: 12 Une école de l’IMT QUASIMODO 2019/11/05

Grouping ■ Reduce redundancy ■ Clustering method based on tri-factorization ■ Groups of (Subject, Object) and Predicate 13 Une école de l’IMT QUASIMODO 2019/11/05

Statistics 14 Une école de l’IMT QUASIMODO 2019/11/05

Examples of facts ■ Practical knowledge from human, e.g. : (car, slip on, ice) ■ Problems linked to a subject, e.g.: (pen, can, leak) ■ Emotions linked to events. e.g.: (divorce, can, hurt) ■ Human behaviors. e.g.: (ghost, scare, people) ■ Negative knowledge, e.g.: Not (elephant, can, jump), ■ Salient modalities, e.g.: Always (doctor, have, unreadable handwriting) ■ Trivial facts, e.g.: (road, has_color, black) ■ Newest facts. e.g.: (trump, build, wall) ■ Cultural knowledge (here U.S.) e.g.: Always (school, have, locker) ■ Comparative knowledge, e.g.: (light, faster than, sound) 15 Une école de l’IMT QUASIMODO 2019/11/05

Precision – Entire CSKs 16 Une école de l’IMT QUASIMODO 2019/11/05

Precision – Same Subjects 17 Une école de l’IMT QUASIMODO 2019/11/05

Recall 18 Une école de l’IMT QUASIMODO 2019/11/05

Question Answering 19 Une école de l’IMT QUASIMODO 2019/11/05

Conclusion ■ We introduced a new methodology for acquiring CSK from non-standard sources ■ Improve state of the art with better coverage of typical and salient properties, determined by Mturks ■ Extrinsic evaluations illustrate advantages ■ Data and code available: github.com/Aunsiels/CSK 20 Une école de l’IMT QUASIMODO 2019/11/05

Additional slides 21 2019/11/05 Une école de l’IMT QUASIMODO

Future Work ■ Cultural knowledge ■ Study of stereotypes ■ Temporal evolution of the knowledge base ■ Improve ranking methods ■ Scale to the entire web 22 Une école de l’IMT QUASIMODO 2019/11/05

Litterature ■ Data: https://www.mpi-inf.mpg.de/departments/databases-and-informatio n-systems/research/yago-naga/commonsense/quasimodo/ ■ Code: https://github.com/Aunsiels/CSK ■ http://conceptnet.io/ ■ http://data.allenai.org/tuple-kb/ ■ https://www.mpi-inf.mpg.de/departments/databases-and-informatio n-systems/research/yago-naga/commonsense/webchild/ 23 Une école de l’IMT QUASIMODO 2019/11/05