machine reading and reasoning with neural program
play

Machine Reading and Reasoning with Neural Program Interpreters - PowerPoint PPT Presentation

Sep 10, 2022 •38 likes •1.61k views

Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro Machine Reading Collaborators Pontus Stenetorp Marzieh Saeidi Max Bartolo Johannes Welbl (Bloomsbury AI) (UCL) (UCL) (Bloomsbury AI)

  1. Machine Reading & Comprehension Which function do I call for storing items in cape? … … Archiving Inbox Items Archiving Inbox Items cape.client.CapeClient.archive_inbox() Once an inbox item has been archived it will no longer Once an inbox item has been archived it will no longer appear in the list of inbox items returned appear in the list of inbox items returned by cape.client.CapeClient.get_inbox(). To archive an by cape.client.CapeClient.get_inbox(). To archive an item call cape.client.CapeClient.archive_inbox() with item call cape.client.CapeClient.archive_inbox() with the ID of the inbox item to archive: the ID of the inbox item to archive: from cape.client import CapeClient from cape.client import CapeClient cc = CapeClient() cc = CapeClient() cc.login('username', 'password') cc.login('username', 'password') cc.archive_inbox('4123') cc.archive_inbox('4123') � 7

  2. How to read and reason end-to-end ? � 8

  3. Learning to Read and Aggregate Which medical specialty deals with pituitary ACTH hypersecretion? � 9

  4. Learning to Read and Aggregate Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … � 9

  5. Learning to Read and Aggregate Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... � 9

  6. Learning to Read and Aggregate Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. � 9

  7. Learning to Read and Aggregate Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. � 9

  8. Part 1 Learning to Read and Aggregate Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. � 9

  9. Part 1 Learning to Read and Aggregate Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders NIPS 2017, TACL 2018 is endocrinology , a branch of internal medicine. � 9

  10. Learning to Read and Calculate Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. � 10

  11. Learning to Read and Calculate Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. 2 � 10

  12. Part 2 Learning to Read and Calculate Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. 2 � 10

  13. Part 2 Learning to Read and Calculate Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. 2 ICML 2017 � 10

  14. Learning to Interpret Rules I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. � 11

  15. Learning to Interpret Rules I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily Yes If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. � 11

  16. Part 3 Learning to Interpret Rules I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily Yes If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. � 11

  17. Part 3 And Ask Clarification Questions I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. � 12

  18. Part 3 And Ask Clarification Questions I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily Have you been working abroad 52 weeks or less? If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. � 12

  19. Part 3 And Ask Clarification Questions I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily Have you been working abroad 52 weeks or less? If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country Yes You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. � 12

  20. Part 3 And Ask Clarification Questions I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily Have you been working abroad 52 weeks or less? If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country Yes You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. Yes � 12

  21. Part 3 And Ask Clarification Questions I am working for an employer in Canada since If you’re self-employed December 2017. Do I need to carry on paying UK You might be able to carry on paying National national insurance? Insurance if you’re: - usually self-employed in the UK - working abroad temporarily Have you been working abroad 52 weeks or less? If you do, you won’t have to pay social security contributions in the country where you’re working. You can check this by completing form CA9107. Working in any other country Yes You’ll carry on paying National Insurance for the first 52 weeks you’re abroad if you’re working for an employer outside the EEA. Yes ONGOING � 12

  22. Pontus Stenetorp Johannes Welbl Tim Rocktäschel Part 1: Learning to Aggregate � 13

  23. Neural Theorem Provers Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. � 14

  24. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. Rocktäschel and Riedel NIPS 2017 � 15

  25. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. Rocktäschel and Riedel NIPS 2017 � 15

  26. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. Reader Rocktäschel and Riedel NIPS 2017 � 15

  27. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Code Question is endocrinology , a branch of internal medicine. [translates questions] Reader Rocktäschel and Riedel NIPS 2017 � 15

  28. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Code Question is endocrinology , a branch of internal medicine. What is ACTH [translates questions] hypersecretion created by? Reader Rocktäschel and Riedel NIPS 2017 � 16

  29. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Code Question is endocrinology , a branch of internal medicine. What is ACTH [translates questions] hypersecretion created by? Reader Rocktäschel and Riedel NIPS 2017 � 16

  30. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Question Code Question is endocrinology , a branch of internal medicine. Which specialty deals What is the pituitary a with pituitary ACTH [translates questions] part of? hypersecretion? Reader Rocktäschel and Riedel NIPS 2017 � 17

  31. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Question Code Question is endocrinology , a branch of internal medicine. Which specialty deals What is the pituitary a with pituitary ACTH [translates questions] part of? hypersecretion? Reader Rocktäschel and Riedel NIPS 2017 � 17

  32. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Question Question Code Question is endocrinology , a branch of internal medicine. Which specialty deals What is the pituitary a with pituitary ACTH What topic covers the [translates questions] part of? hypersecretion? endocrine system? Reader Rocktäschel and Riedel NIPS 2017 � 18

  33. Neural Theorem Provers Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Question Question Code Question is endocrinology , a branch of internal medicine. Which specialty deals What is the pituitary a with pituitary ACTH What topic covers the [translates questions] part of? hypersecretion? endocrine system? Reader Rocktäschel and Riedel NIPS 2017 � 18

  34. Vectors Correspond to Interpretable Rules Pituitary ACTH hypersecretion ... is a form of Which medical specialty deals with pituitary hyperpituitarism characterized by an abnormally high ACTH hypersecretion? level of ACTH produced by the anterior pituitary … Agent A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... Program Interpreter The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders Question Question Code Question is endocrinology , a branch of internal medicine. Which specialty deals X deals with Y if What is the pituitary a with pituitary ACTH What topic covers the Y produced by Z part of? Z is a part of U hypersecretion? endocrine system? X deals with U Reader Rocktäschel and Riedel NIPS 2017 � 19

  35. Catch: Currently only Works on Relational Data dealsWith( hypersecretion , X) createdBy( hypersecretion, anterior pituitary ) Agent partOf( endocrine system , anterior pituitary ) Program Interpreter dealsWith( endocrine system , endocrinology ) Question Question Code Question Which specialty deals X deals with Y if What is the pituitary a with pituitary ACTH Y produced by Z covers( endocrine part of? Z is a part of U hypersecretion? system , X ) X deals with U Reader Rocktäschel and Riedel NIPS 2017 � 20

  36. Why Program Interpreter/Theorem Prover? dealsWith( hypersecretion , X) createdBy( hypersecretion, anterior pituitary ) Agent partOf( endocrine system , anterior pituitary ) Amounts the Backward dealsWith( endocrine system , endocrinology ) Chaining algorithm used in Prolog Interpreters Rocktäschel and Riedel NIPS 2017 � 21

  37. Why Neural? Supports Soft Unification dealsWith( hypersecretion , X) createdBy( hypersecretion, anterior pituitary ) Agent partOf( endocrine system , anterior pituitary ) Program Interpreter dealsWith ( endocrine system , endocrinology ) Question Question Code Question Which specialty deals X deals with Y if What is the pituitary a with pituitary ACTH Y produced by Z covers ( endocrine part of? Z is a part of U hypersecretion? system , X ) X covers U Reader Rocktäschel and Riedel NIPS 2017 � 22

  38. Why Neural? Supports Soft Unification dealsWith( hypersecretion , X) createdBy( hypersecretion, anterior pituitary ) Agent partOf( endocrine system , anterior pituitary ) Program Interpreter dealsWith ( endocrine system , endocrinology ) Question Question Code Question Which specialty deals X deals with Y if What is the pituitary a with pituitary ACTH Y produced by Z covers ( endocrine part of? Z is a part of U hypersecretion? system , X ) X covers U Reader Rocktäschel and Riedel NIPS 2017 � 22

  39. Learn Rules by Learning Symbol Representations dealsWith( hypersecretion , X) createdBy( hypersecretion, anterior pituitary ) Agent partOf( endocrine system , anterior pituitary ) Program Interpreter dealsWith ( endocrine system , endocrinology ) Question Question Code Question Which specialty deals X rel32 Y if What is the pituitary a with pituitary ACTH Y rel42 Z rel62 ( endocrine part of? Z rel51 U hypersecretion? system , X ) X rel62 U Reader Rocktäschel and Riedel NIPS 2017 � 23

  40. Before Learning X rel32 Y if Y rel42 Z Z rel51 U is a part of rel32 X rel62 U produced by rel62 covers rel51 X deals with Y if rel42 Y produced by Z Z is a part of U deals with X deals with U � 24

  41. After Learning X rel32 Y if Y rel42 Z Z rel51 U X rel62 U produced by rel42 is a part of rel51 rel32 rel62 X deals with Y if Y produced by Z covers deals with Z is a part of U X deals with U � 25

  42. Results on Benchmark (Rank of Correct Answer) Rocktäschel and Riedel NIPS 2017 � 26

  43. Results on Benchmark (Rank of Correct Answer) 100 75 50 25 0 Complex NTP Complex NTP Complex NTP Countries S3 UMLS Nations Rocktäschel and Riedel NIPS 2017 � 26

  44. Results on Benchmark (Rank of Correct Answer) 100 96 86 75 50 48.4 25 0 Complex NTP Complex NTP Complex NTP Countries S3 UMLS Nations Rocktäschel and Riedel NIPS 2017 � 26

  45. Results on Benchmark (Rank of Correct Answer) 100 99 96 89 86 75 77.3 50 48.4 25 0 Complex NTP Complex NTP Complex NTP Countries S3 UMLS Nations Rocktäschel and Riedel NIPS 2017 � 26

  46. Results on Benchmark (Rank of Correct Answer) 100 99 96 89 86 75 77.3 50 48.4 25 0 Complex NTP Complex NTP Complex NTP Countries S3 UMLS Nations Comparable or Better than Baselines, and interpretable Rocktäschel and Riedel NIPS 2017 � 26

  47. Limitations Scalability Currently only works for KBs with < 10k facts (solved!) Small proof depth (getting there) Still requires relational representation (ongoing) � 27

  48. Pontus Stenetorp Johannes Welbl How to create Datasets for Multihop Reading and Reasoning? � 28

  49. Dataset Construction Method TACL 2018 � 29

  50. Dataset Construction Method Unlabelled Text Knowledge Base TACL 2018 � 29

  51. Dataset Construction Method Unlabelled Text Dataset Construction Method Knowledge Base TACL 2018 � 29

  52. Dataset Construction Method Unlabelled Text Dataset Construction Method Multihop Dataset Knowledge Base TACL 2018 � 29

  53. Dataset Construction Method KB Triple Documents Instance Entities � 30

  54. Dataset Construction Method KB Triple Documents Instance Entities Jamie Burnett, citizenship, Scotland � 30

  55. Dataset Construction Method KB Triple Documents Instance Entities What is the nationality of Jamie Burnett? Jamie Burnett, citizenship, Scotland � 30

  56. Dataset Construction Method KB Triple Documents Instance Entities What is the nationality of Jamie Burnett? Jamie Burnett, citizenship, Scotland � 30

  57. Dataset Construction Method KB Triple Documents Instance Entities What is the nationality of Jamie Burnett? Jamie Burnett Jamie Burnett, citizenship, Scotland � 30

  58. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett Jamie Burnett, citizenship, Scotland � 30

  59. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett Jamie Burnett, citizenship, Scotland � 30

  60. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Jamie Burnett, citizenship, Scotland � 30

  61. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland � 30

  62. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland described in � 30

  63. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland described in � 30

  64. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland described in mentions � 30

  65. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in mentions � 30

  66. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in mentions � 30

  67. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in Yixing Open mentions � 30

  68. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in Yixing Open mentions � 30

  69. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in Yixing Open mentions � 30

  70. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in Yixing Open mentions � 30

  71. Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in China Yixing Open mentions China � 30

  72. Dataset Construction Method Unlabelled Text Dataset Construction Method Multihop Dataset Knowledge Base [Conditionally Accepted to TACL] � 31

  73. Dataset Construction Method Unlabelled Text Dataset Construction Method WikiHop Knowledge Base � 32

  74. Dataset Construction Method Unlabelled Text Dataset Construction Method MedHop Knowledge Base � 33

  75. Baseline Results � 34

  76. Baseline Results 50 Random TF-IDF BiDAF 41.667 33.333 Accuracy [%] 25 16.667 8.333 0 WikiHop MedHop � 34

  77. Baseline Results 50 Random TF-IDF BiDAF 41.667 33.333 Accuracy [%] 25 16.667 13.9 11.5 8.333 0 WikiHop MedHop � 34

  78. Baseline Results 50 Random TF-IDF BiDAF 41.667 33.333 Accuracy [%] 25 25.6 16.667 13.9 11.5 8.333 9 0 WikiHop MedHop � 34

  79. Baseline Results 50 Random 47.8 TF-IDF BiDAF 42.9 41.667 33.333 Accuracy [%] 25 25.6 16.667 13.9 11.5 8.333 9 0 WikiHop MedHop � 34

  80. Reduction to Traditional Machine Comprehension Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. � 35

  81. Reduction to Traditional Machine Comprehension Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. � 36

  82. Do Neural Reading Models Aggregate? Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. � 37

  83. Removing Relevant Documents, Keep Answer Documents 55 54.5 BIDAF 10% BIDAF doc-rem 50 45 44.6 Accuracy [%] 40 35 33.7 30 30.4 25 WikiHop MedHop � 38

  84. TACL 2018 � 39

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro

Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro

Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro Bloomsbury AI Machine Reading Collaborators Pontus Stenetorp Matko Bosnjak Johannes Welbl (UCL) (UCL) (UCL) Tim Rocktschel Jason Naradowsky

814 views • 68 slides
Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro

Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro

Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro Bloomsbury AI Machine Reading Collaborators Pontus Stenetorp Matko Bosnjak Johannes Welbl (UCL) (UCL) (UCL) Tim Rocktschel Jason Naradowsky

770 views • 49 slides
Automated Reasoning (what this course is about) AUTOMATED REASONING machine does 'thinking'

Automated Reasoning (what this course is about) AUTOMATED REASONING machine does 'thinking'

0ai Automated Reasoning (what this course is about) AUTOMATED REASONING machine does 'thinking' machine does book-keeping, Krysia Broda user does nothing user does 'thinking'. Room 378 Concerned with GENERAL DEDUCTION - applicable in many

390 views • 5 slides
2nd Session Machine learning: feed-forward neural networks and self-organizing maps 1

2nd Session Machine learning: feed-forward neural networks and self-organizing maps 1

2nd Session Machine learning: feed-forward neural networks and self-organizing maps 1 Recommended reading J. Zupan, J. Gasteiger, Neural Networks in Chemistry and Drug Design: An Introduction, Wiley-VCH, Weinheim, 1999.

959 views • 62 slides
Neural Machine Translation Gongbo Tang 8 October 2018 Outline Neural Machine Translation 1

Neural Machine Translation Gongbo Tang 8 October 2018 Outline Neural Machine Translation 1

Neural Machine Translation Gongbo Tang 8 October 2018 Outline Neural Machine Translation 1 Advances and Challenges 2 Gongbo Tang Neural Machine Translation 2/52 Neural Machine Translation Figure Recurrent neural network based NMT

906 views • 73 slides
Overfitting, Cross-Validation Recommended reading: Neural nets: Mitchell Chapter 4

Overfitting, Cross-Validation Recommended reading: Neural nets: Mitchell Chapter 4

Overfitting, Cross-Validation Recommended reading: Neural nets: Mitchell Chapter 4 Decision trees: Mitchell Chapter 3 Machine Learning 10-701 Tom M. Mitchell Carnegie Mellon University Overview Followup on neural networks

439 views • 24 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides
Probabilistic Reasoning; Probabilistic Reasoning; Network-based reasoning Network-based

Probabilistic Reasoning; Probabilistic Reasoning; Network-based reasoning Network-based

Probabilistic Reasoning; Probabilistic Reasoning; Network-based reasoning Network-based reasoning COMPSCI 276, Fall 2014 Set 1: Introduction and Background Rina Dechter (Reading: Pearl chapter 1-2, Darwiche chapters 1,3) 1 Why/What/How

700 views • 45 slides
Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning Motivation NS-RL Neural Symbolic Input Prediction Error Network Reasoning Ground Truth Forward pass

1.13k views • 27 slides
Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks These are the commonest type of neural

534 views • 32 slides
Automated Reasoning (what this course is about) machine does 'thinking' user does 'thinking'

Automated Reasoning (what this course is about) machine does 'thinking' user does 'thinking'

0ai Automated Reasoning (what this course is about) machine does 'thinking' user does 'thinking' AUTOMATED REASONING user does nothing machine keeps the books Krysia Broda Concerned with GENERAL DEDUCTION - applicable in many areas.

357 views • 6 slides
Neural Machine Translation Philipp Koehn 6 October 2020 Philipp Koehn Machine Translation:

Neural Machine Translation Philipp Koehn 6 October 2020 Philipp Koehn Machine Translation:

Neural Machine Translation Philipp Koehn 6 October 2020 Philipp Koehn Machine Translation: Neural Machine Translation 6 October 2020 Language Models 1 Modeling variants feed-forward neural network recurrent neural network long

485 views • 36 slides
Convolutional over Recurrent Encoder for Neural Machine Translation Praveen Dakwale and Christof

Convolutional over Recurrent Encoder for Neural Machine Translation Praveen Dakwale and Christof

Annual Conference of the European Association for Machine Translation 2017 Convolutional over Recurrent Encoder for Neural Machine Translation Praveen Dakwale and Christof Monz Neural Machine Translation End to end neural network with RNN

354 views • 20 slides
10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview Clustering - Hierarchical, semi-supervised learning Graphical models - Bayesian networks, HMMs, Reasoning under uncertainty Putting it

862 views • 44 slides
Machine Learning for NLP Neural networks and neuroscience Aurlie Herbelot 2018 Centre for

Machine Learning for NLP Neural networks and neuroscience Aurlie Herbelot 2018 Centre for

Machine Learning for NLP Neural networks and neuroscience Aurlie Herbelot 2018 Centre for Mind/Brain Sciences University of Trento 1 Introduction 2 Towards an integration of deep learning and neuroscience Today: reading

775 views • 52 slides
Reasoning with Neural Networks Rodrigo Toro Icarte (rntoro@cs.toronto.edu) March 08, 2016

Reasoning with Neural Networks Rodrigo Toro Icarte (rntoro@cs.toronto.edu) March 08, 2016

University of Toronto Faculty of Arts and Science Department of Computer Science Reasoning with Neural Networks Rodrigo Toro Icarte (rntoro@cs.toronto.edu) March 08, 2016 Introduction Reasoning with Neural Networks Questions References

750 views • 61 slides
J-PARC High Intensity Neutrino Beam T. Sekiguchi (KEK) on behalf of T2K Beam Group Contents

J-PARC High Intensity Neutrino Beam T. Sekiguchi (KEK) on behalf of T2K Beam Group Contents

J-PARC High Intensity Neutrino Beam T. Sekiguchi (KEK) on behalf of T2K Beam Group Contents Introduction to J-PARC Neutrino Beamline Current Status Prospect for Beamline Upgrade 2 J-PARC Hadron Experimental Facility

570 views • 32 slides
Dunk Island Work Experience work! We joined the rest of the staff in June 1 - 7, 2008 the

Dunk Island Work Experience work! We joined the rest of the staff in June 1 - 7, 2008 the

Issue 08, June 20, 2008 quickly gaining THOUGHT FOR THE DAY confidence with the There is no limit to what can be accomplished if it doesn't routines and the matter who gets the credit - Ralph Waldo Emerson guests. What a beautiful

995 views • 4 slides
ECONOMICS HSC EC HEAD START LECTURE 2019 Presented by: Isaac Dela Torre Todays Agenda

ECONOMICS HSC EC HEAD START LECTURE 2019 Presented by: Isaac Dela Torre Todays Agenda

ECONOMICS HSC EC HEAD START LECTURE 2019 Presented by: Isaac Dela Torre Todays Agenda Economic Skills Globalisation HOW TO SMASH HSC ECONOMICS!! Protection Development Case Study Introduction Housekeeping Rules Ask Questions

1.02k views • 83 slides
Metaprogramming Ruby - What the hell's a DSL? murphee (Werner Schuster) Blog @

Metaprogramming Ruby - What the hell's a DSL? murphee (Werner Schuster) Blog @

Metaprogramming Ruby - What the hell's a DSL? murphee (Werner Schuster) Blog @ http://jroller.com/page/murphee Meta? Programs that write Programs Compiler DSLs Regexes, anyone? Always good to start with quotes by... Alan Kay The

752 views • 58 slides
HOME Homebuyer Programs &amp; Policies January 14, 2020 Presented by: Monte Franke

HOME Homebuyer Programs &amp; Policies January 14, 2020 Presented by: Monte Franke

HOME Homebuyer Programs &amp; Policies January 14, 2020 Presented by: Monte Franke MLFranke@aol.com PJ Homebuyer Policies &amp; P rocedures HUD requirements 92.254(f): PJs must have homebuyer program policies/procedures:

439 views • 24 slides
Homebuyer Program Policies and Procedures January 8, 2018 Welcome &amp; Introductions

Homebuyer Program Policies and Procedures January 8, 2018 Welcome &amp; Introductions

Homebuyer Program Policies and Procedures January 8, 2018 Welcome &amp; Introductions Sponsored by HUDs Office of Affordable Housing Programs NCSHA Trainer Steve Lathom, TDA Consulting slathom@tdainc.org 5172034130

385 views • 23 slides
Becoming a CHDO: Program Requirements for Successfully Obtaining CHDO Designation RU R A L H O U S

Becoming a CHDO: Program Requirements for Successfully Obtaining CHDO Designation RU R A L H O U S

Becoming a CHDO: Program Requirements for Successfully Obtaining CHDO Designation RU R A L H O U S I N G C A PAC I T Y B U I L D I N G PRO G R A M ( R H C B ) F U N D E D BY T H E D E PA RT M E N T O F H E A LT H A N D H U M A N S E RV I C E S CO M

650 views • 28 slides
CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar

CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar

CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar Formalisms Yejin Choi - University of Washington Dependency Grammar For each word, find one parent. Child Parent A child is dependent on the

733 views • 56 slides

More recommend