Open Question Answering Over Curated and Extracted Knowledge Bases - - PowerPoint PPT Presentation

Open Question Answering Over Curated and Extracted Knowledge Bases

Anthony Fader, Luke Zettlemoyer, Oren Etzioni Allen Institute for AI

Presented by: Yashoteja Prabhu Slide Credits: Gareth Dwyer, Paper Authors

Open-Domain Question-Answering

• No domain restriction on the questions
• Unlike closed domain QnA
• Cannot rely too much on
• FAQ collection
• Fixed set of books/documents
• Domain knowledge derived QnA templates
• Natural language queries
• Same question can be asked in numerous ways:

How can I tell if I have strep throat ? What are the signs of strep throat ?

Closed Domain

Current Work: An Example Derivation

10 hand-written precise templates

• Too many ways of asking the same question
• Convert to equivalent questions which are

easy to parse

• Also Wh-questions are easier to answer
• Parsing: Convert to KB-friendly conjunction of

relations

• Uses small and fixed set of templated
• Multiple equivalent relations exist
• KB only contains some of those
• Rewrite to equivalent relation present in KB
• Search the relation in KB to obtain answers

= log 𝑞(𝑦, 𝑧) 𝑞 𝑦 𝑞(𝑧)

Features:

• Indicator function for the pattern used
• POS sequence of matched arguments

Discussion - Pros

• Breaking into subproblems is advantageous and improves P-R [Barun, Dinesh,

Surag, Arindam, Anshul]

• Thorough ablation study [Barun, Nupur, Rishabh, Shantanu]
• Lot of examples (including negative ones) [Gagan, Haroun]
• Unlexicalised approach is advantageous [Gagan]
• Indirect supervision using QA pairs is nice [Arindam]
• Multiple KBs [Rishabh]

Discussion - Cons

• 10 handwritten templates not enough in parsing [Barun, Anshul, Prachi]
• Handling of no derivation case [Barun, Dinesh]
• Query rewriting seems useless [Barun]
• Correct but lengthy derivations are wrongly penalized [Surag]
• Beam search uses same parameter for all derivation steps [Anshul]

Discussion - Questions

• Shouldn't query-rewriting and paraphrasing compensate for the lack of lexical

features in the system? [Nupur]

• What happens if the high confidence gold answer set is incomplete, and the

knowledge base has the answer and system figure out the answer? I think this would be more valid for WikiAnswers where are no answers [Rishabh]

• Others ?!

Discussion - Extensions

• Word/Phrase/deep embeddings [Surag, Barun, Arindam, Rishabh, Prachi]
• RL instead of beam search [Barun]
• Perceptron could be based on approx. answer matching [Gagan]
• Extension to True/False type answers [Gagan]
• Combining KB and search engine for better recall [Rishabh]
• Modifying scoring function to be non-linear