Open Question Answering Over Curated and Extracted Knowledge Bases - PowerPoint PPT Presentation

Open Question Answering Over Curated and Extracted Knowledge Bases Anthony Fader, Luke Zettlemoyer, Oren Etzioni Allen Institute for AI Presented by: Yashoteja Prabhu Slide Credits: Gareth Dwyer, Paper Authors

Open-Domain Question-Answering • No domain restriction on the questions • Unlike closed domain QnA • Cannot rely too much on - FAQ collection - Fixed set of books/documents - Domain knowledge derived QnA templates • Natural language queries • Same question can be asked in numerous ways: How can I tell if I have strep throat ? What are the signs of strep throat ?

Closed Domain

Current Work: An Example Derivation • Too many ways of asking the same question • Convert to equivalent questions which are easy to parse • Also Wh-questions are easier to answer • Parsing: Convert to KB-friendly conjunction of 10 hand-written precise templates relations • Uses small and fixed set of templated • Multiple equivalent relations exist • KB only contains some of those • Rewrite to equivalent relation present in KB • Search the relation in KB to obtain answers

𝑞(𝑦, 𝑧) = log 𝑞 𝑦 𝑞(𝑧)

Features: • Indicator function for the pattern used • POS sequence of matched arguments

Discussion - Pros • Breaking into subproblems is advantageous and improves P-R [Barun, Dinesh, Surag, Arindam, Anshul] • Thorough ablation study [Barun, Nupur, Rishabh, Shantanu] • Lot of examples (including negative ones) [Gagan, Haroun] • Unlexicalised approach is advantageous [Gagan] • Indirect supervision using QA pairs is nice [Arindam] • Multiple KBs [Rishabh]

Discussion - Cons • 10 handwritten templates not enough in parsing [Barun, Anshul, Prachi] • Handling of no derivation case [Barun, Dinesh] • Query rewriting seems useless [Barun] • Correct but lengthy derivations are wrongly penalized [Surag] • Beam search uses same parameter for all derivation steps [Anshul]

Discussion - Questions • Shouldn't query-rewriting and paraphrasing compensate for the lack of lexical features in the system? [Nupur] • What happens if the high confidence gold answer set is incomplete, and the knowledge base has the answer and system figure out the answer? I think this would be more valid for WikiAnswers where are no answers [Rishabh] • Others ?!

Discussion - Extensions • Word/Phrase/deep embeddings [Surag, Barun, Arindam, Rishabh, Prachi] • RL instead of beam search [Barun] • Perceptron could be based on approx. answer matching [Gagan] • Extension to True/False type answers [Gagan] • Combining KB and search engine for better recall [Rishabh] • Modifying scoring function to be non-linear