scott wen tau yih what are the names of obama s daughters
play

Scott Wen-tau Yih # , What are the names of Obamas daughters? Q: - PowerPoint PPT Presentation

Scott Wen-tau Yih # , What are the names of Obamas daughters? Q: Who won the best actor Oscar in 1973? S 1 : Jack Lemmon was awarded the Best Actor Oscar for Save the Tiger (1973). S 2 : Academy award winner Kevin Spacey said that Jack


  1. Scott Wen-tau Yih # ,

  2. “ What are the names of Obama’s daughters? ”

  3. Q: Who won the best actor Oscar in 1973? S 1 : Jack Lemmon was awarded the Best Actor Oscar for Save the Tiger (1973). S 2 : Academy award winner Kevin Spacey said that Jack Lemmon is remembered as always making time for others.

  4. • Question distribution • Candidate selection bias Q: How did Seminole war end? A: Ultimately, the Spanish Crown ceded the colony to United States rule. • Excluding questions that have no correct answers

  5. • Question distribution • Candidate selection • Including questions that have no correct answers

  6. • Introduction • WikiQA Dataset • Data construction and annotation • Data statistics

  7. “ Who wrote second Corinthians? ”

  8. • Step 1: Does the short paragraph answer the question? Question: Who wrote second Corinthians? Second Epistle to the Corinthians The Second Epistle to the Corinthians, often referred to as Second Corinthians (and written as 2 Corinthians), is the eighth book of the New Testament of the Bible. Paul the Apostle and “Timothy our brother” wrote this epistle to “the church of God which is at Corinth, with all the saints which are in all Achaia”.

  9. • Step 2: Check all the sentences that can answer the question in isolation (assuming coreferences and pronouns have been resolved correctly) Question: Who wrote second Corinthians? Second Epistle to the Corinthians q The Second Epistle to the Corinthians, often referred to as Second Corinthians (and written as 2 Corinthians), is the eighth book of the New Testament of the Bible. q Paul the Apostle and “Timothy our brother” wrote this epistle to “the church of God which is at Corinth, with all the saints which are in all Achaia”.

  10. 3,047 3,500 3,000 2,500 13.4x 2,000 1,500 1,000 227 500 - # of questions QASent WikiQA

  11. 35,000 29,258 30,000 3.5x 25,000 20,000 15,000 8,478 10,000 3,047 5,000 227 - # of questions # of sentences QASent WikiQA

  12. QASent WikiQA Abbr. Desc. Abbr. Entity Entity 1% 7% 1% 14% 16% Desc. Locatio 35% Human n Locatio 29% 12% n 16% Numeri Numeri c Human c 22% 16% 31%

  13. • Introduction • WikiQA Dataset • Experiments • Baseline systems • Evaluation on answer sentence selection • Evaluation metric & results on answer triggering

  14. • Word matching count (Wd-Cnt) • # non-stopwords in Q that also occur in S • Latent word alignment (Wd-Algn) [Yih+ ACL-13] 𝑟 : What is the fastest car in the world? 𝑔(𝑟 , 𝑡) = ​𝜄↑𝑈 Φ( ℎ ) 𝒊 𝑡 : The Jaguar XJ220 is the dearest, fastest and most sought after car on the planet. • Convolutional NN (CNN) [Yu+ DLWorkshop-14] • Convolutional NN & Wd-Cnt (CNN-Cnt)

  15. 0.9 0.7617 0.7633 Mean Reciprocal Rank (MRR) 0.8 0.6662 0.7 0.623 0.6 0.5 0.4 0.3 0.2 0.1 0 QASent Wd-Cnt Wd-Algn CNN CNN-Cnt

  16. 0.9 0.7617 0.7633 Mean Reciprocal Rank (MRR) 0.8 0.6662 0.6652 0.7 0.6281 0.623 0.6086 0.6 0.4924 0.5 0.4 0.3 0.2 0.1 0 QASent WikiQA* Wd-Cnt Wd-Algn CNN CNN-Cnt

  17. • Data – All questions in WikiQA are included in this task

  18. 33 32.5 Question-level F1 score 32 31.5 31 30.61 30.5 30 29.5 29 28.5 28 CNN-Cnt +Q-Length +S-Length +Q-Class +All

  19. 33 32.5 32.17 Question-level F1 score 32 31.5 31 30.61 30.5 30 29.5 29 28.5 28 CNN-Cnt +Q-Length +S-Length +Q-Class +All

  20. 33 32.5 32.17 Question-level F1 score 32 31.64 31.5 30.92 31 30.61 30.34 30.5 30 29.5 29 28.5 28 CNN-Cnt +Q-Length +S-Length +Q-Class +All

  21. http://aka.ms/WikiQA • Includes “answer phrases” labeled by authors

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend