recent advances in automated fact checking
play

Recent Advances in Automated Fact Checking Immanuel Trummer - PowerPoint PPT Presentation

Jul 13, 2023 •538 likes •1.5k views

Recent Advances in Automated Fact Checking Immanuel Trummer Cornell University Automation & Fact Checking Automation & Fact Checking Lorem Ipsum ... Identifying Lorem Ipsum ... Check-Worthy Claims

  1. Recent Advances 
 in Automated Fact Checking Immanuel Trummer 
 Cornell University

  2. Automation & 
 Fact Checking

  3. Automation & 
 Fact Checking Lorem 
 Ipsum ... Identifying 
 Lorem 
 Ipsum ... Check-Worthy 
 Claims Lorem 
 Ipsum ... Lorem 
 Ipsum ... Claim Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ...

  4. Automation & 
 Fact Checking Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Identifying 
 Ipsum ... Lorem 
 Ipsum ... Check-Worthy 
 Claims Lorem 
 Lorem 
 Ipsum ... Lorem Ipsum ... Lorem 
 Ipsum ... Ipsum ... Claim Check Matching 
 Lorem 
 Claims to 
 Ipsum ... Lorem 
 Ipsum ... Checks Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ...

  5. Automation & 
 Fact Checking Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Identifying 
 Ipsum ... Lorem 
 Ipsum ... Check-Worthy 
 Claims Lorem 
 Verification Lorem 
 Ipsum ... Lorem Ipsum ... Lorem 
 Ipsum ... Ipsum ... Claim Check Matching 
 Lorem 
 Claims to 
 Ipsum ... Lorem 
 Ipsum ... Checks Talk Focus Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ...

  6. Data-Driven Fact Checking Claim Data Formula Verified/Refuted

  7. Data-Driven Fact Checking Claim Data Formula (aka. SQL Query ) Verified/Refuted

  8. Data-Driven Fact Checking Claim Data Formula Verified/Refuted

  9. Data-Driven Fact Checking (In Natural Language) Claim Data Formula Verified/Refuted

  10. Data-Driven Fact Checking (In Natural Language) Claim ? a t a d h c i h W Data Formula Verified/Refuted

  11. Data-Driven Fact Checking (In Natural Language) Claim ? a ? l a u t m a d r o h f c h i h c W i h W Data Formula Verified/Refuted

  12. International Energy Agency

  13. International Energy Agency Paris-based intergovernmental organization Established in 1974, 30 member countries Mission: serve statistics on energy sector

  14. International Energy Agency Paris-based intergovernmental organization Established in 1974, 30 member countries Mission: serve statistics on energy sector

  16. Claims Marked Up

  17. Claims Marked Up Hundreds of pages, Thousands of claims ...

  18. Claims Marked Up Verification Takes Weeks! Hundreds of pages, Thousands of claims ...

  19. Fact Checking @ IEA Claim Data Formula Verified/Refuted

  20. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Verified/Refuted

  21. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 Electricity/Global/2016 Verified/Refuted

  22. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 D17/D16=1.03 Electricity/Global/2016 Verified/Refuted

  23. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 D17/D16=1.03 Electricity/Global/2016 Verified/Refuted

  24. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 D17/D16=1.03 Electricity/Global/2016 Verified/Refuted

  25. The "Infodemic"

  26. Fighting the Infodemic

  27. Fighting the Infodemic

  28. Fighting the Infodemic

  29. Fighting the Infodemic France has 
 more cases than US Claim Data Formula CDC/Confirmed/France F>U CDC/Confirmed/US Verified/Refuted

  30. Fighting the Infodemic France has 
 more cases than US Claim Data Formula CDC/Confirmed/France F>U CDC/Confirmed/US Verified/Refuted

  31. Fighting the Infodemic France has 
 more cases than US Claim Data Formula CDC/Confirmed/France F>U CDC/Confirmed/US Verified/Refuted

  32. Other Use Cases • Data journalism • Business reports • Scientific papers • ...

  33. (Demo) Verifying text summaries of relational data sets. 
 SIGMOD 2019 
 S. Jo, I. Trummer, W. Yu, X. Wang, C. Yu, D. Liu, N. Mehta.

  34. Challenges

  35. Challenges Text-Data Inconsistency ... ... American ...

  36. Challenges Multi-Claim Sentences Text-Data Inconsistency ... ... American ...

  37. Challenges Context Multi-Claim Sentences Text-Data Inconsistency ... ... American ...

  38. Fully Automated Checking Claim Translation Formula Evaluation

  39. Fully Automated Checking Claim May go Wrong! Translation Formula Evaluation

  40. Semi-Automated Checking Claim Translation Probability Formula Evaluation

  41. Analyze Data Structure country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  42. Analyze Data Structure country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  43. Analyze Data Structure Germany Beer Spirit Keywords Serving Country USA ... country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  44. Analyze Data Structure United States Synonyms ... America ... U.S. Germany Beer Spirit Keywords Serving Country USA ... country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  45. Analyze Data Structure Matches United States Synonyms ... America ... U.S. Germany Beer Spirit Keywords Serving Country USA ... country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  46. Analyze Sentence Structure

  47. Analyze Sentence Structure

  48. Analyze Sentence Structure

  49. Analyze Sentence Structure

  50. Consider Text Structure Claim Sentence

  51. Consider Text Structure Paragraph Claim Sentence

  52. Consider Text Structure Section Paragraph Claim Sentence

  53. Consider Text Structure Chapter Section Paragraph Claim Sentence

  54. Consider Text Structure Chapter Section Paragraph Claim Sentence Integrate Surrounding Keywords

  55. Understand the Author Translate Text

  56. Understand the Author Translate Text Claim 
 Translation 
 Hypothesis Infer Topic

  57. Understand the Author Translate Text Document 
 Claim 
 Topic 
 Translation 
 Hypothesis Hypothesis Infer Topic

  58. (Demo) Verifying text summaries of relational data sets. 
 SIGMOD 2019 
 S. Jo, I. Trummer, W. Yu, X. Wang, C. Yu, D. Liu, N. Mehta.

  59. System Overview Lorem 
 Ipsum ... Data Analysis Text Analysis Topic Analysis

  60. Automated Accuracy 100 Correctness Chance 75 50 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  61. Automated Accuracy (Billions of possible formulas) 100 Correctness Chance 75 50 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  62. Automated Accuracy (Billions of possible formulas) 100 Correctness Chance 75 50 Want Auto-Suggestions 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  63. Automated Accuracy (Billions of possible formulas) 100 Need Human Feedback Correctness Chance 75 50 Want Auto-Suggestions 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  64. User Study Results With Tool Without Tool 1.6 Claims per Minute 1.2 6x Speedup! 0.8 0.4 0 Verification Method

  65. Mistakes Discovered 11% of claims were incorrect 7% average error of claim value Analyzed 50 articles from major data journalism venues

  66. Scaling It Up Lorem 
 Ipsum ... AggChecker

  67. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... AggChecker

  68. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... AggChecker

  69. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... AggChecker

  70. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... Scrutinizer

  71. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ...

  72. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ...

  73. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... Learning Data Mapper

  74. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... ! k s A Ipsum ... Learning Data Mapper n r a e L

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Cure My FEVER: Building, Breaking, and Fixing Models for Fact-Checking Christopher Hidey Tuhin

Cure My FEVER: Building, Breaking, and Fixing Models for Fact-Checking Christopher Hidey Tuhin

Cure My FEVER: Building, Breaking, and Fixing Models for Fact-Checking Christopher Hidey Tuhin Chakrabarty Tariq Alhindi Siddharth Varia Kriste Krstovski Mona Diab Smaranda Muresan Automated Fact-checking and Related Tasks Source

382 views • 36 slides
Automated Reasoning LTL Model Checking Alan Bundy Automated Reasoning LTL Model Checking

Automated Reasoning LTL Model Checking Alan Bundy Automated Reasoning LTL Model Checking

Automated Reasoning LTL Model Checking Alan Bundy Automated Reasoning LTL Model Checking Lecture 9, page 1 Introduction So far we have looked at theorem proving Powerful, especially where good sets of rewrite rules or decision procedures

639 views • 26 slides
Recent Advances in Causality Checking Florian Leitner-Fischer University of

Recent Advances in Causality Checking Florian Leitner-Fischer University of

Recent Advances in Causality Checking Florian Leitner-Fischer University of Konstanz Department of Computer and Information Science Chair for Software Engineering software software engineering engineering Joint work

819 views • 26 slides
Recent Advances in Photonic Recent Advances in Photonic effect employing IP- based distributed

Recent Advances in Photonic Recent Advances in Photonic effect employing IP- based distributed

Development of Photonic Network Optical burst switching Next Generation Enhancement of Networks Dynamic (Adaptive) Optical Path Control statistical >10Tb/s multiplexing Photonic MPLS Network Recent Advances in Photonic Recent Advances

762 views • 8 slides
Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances in Biomolecular NMR Protonless NMR for the characterization of Unfolded proteins, Large protein assemblies, Paramagnetic systems I n cell

1.23k views • 70 slides
Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances in Biomolecular NMR Protonless NMR for the characterization of Unfolded proteins, Large protein assemblies, Paramagnetic systems I n cell

635 views • 42 slides
The Mobius Program Verification Environment Recent Advances in Extended Static Checking Joe

The Mobius Program Verification Environment Recent Advances in Extended Static Checking Joe

The Mobius Program Verification Environment Recent Advances in Extended Static Checking Joe Kiniry, University College Dublin Joe Kiniry, University College Dublin http://mobius.inria.fr/ http://mobius.inria.fr/ Contract n IST 015905

534 views • 21 slides
Automated Reasoning Model Checking with Spin (III) Alan Bundy Automated Reasoning Model

Automated Reasoning Model Checking with Spin (III) Alan Bundy Automated Reasoning Model

Automated Reasoning Model Checking with Spin (III) Alan Bundy Automated Reasoning Model Checking with Spin (III) Lecture 12, page 1 Outline of Verification Process Objective: form one big automata, which Spin can use for verification.

1.01k views • 25 slides
Automated Reasoning Model Checking with SPIN (II) Alan Bundy Automated Reasoning SPIN (II)

Automated Reasoning Model Checking with SPIN (II) Alan Bundy Automated Reasoning SPIN (II)

Automated Reasoning Model Checking with SPIN (II) Alan Bundy Automated Reasoning SPIN (II) page 1 Verifying Global Properties Assertions can be used to verify a property locally locally For example, place assert(MemReturned) at the

441 views • 25 slides
Automated Reasoning Model Checking with SPIN Alan Bundy Automated Reasoning SPIN Lecture 10,

Automated Reasoning Model Checking with SPIN Alan Bundy Automated Reasoning SPIN Lecture 10,

Automated Reasoning Model Checking with SPIN Alan Bundy Automated Reasoning SPIN Lecture 10, page 1 Model Checking in Practice System behaviour: what is possible in our model? We will use the language Promela to specify behaviour

341 views • 18 slides
Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me...

Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me...

Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me... Research Interests: Postdoc Carnegie Mellon University, USA Automated Program Repair w/ Claire Le Goues Fault Localisation GI for

676 views • 39 slides
Confl flict and Development: Recent Advances and Future Agendas Professor Patricia Justino

Confl flict and Development: Recent Advances and Future Agendas Professor Patricia Justino

Confl flict and Development: Recent Advances and Future Agendas Professor Patricia Justino Institute of Development Studies, UK Where we are Three main advances in recent research on conflict and violence: shift from state-level to more

323 views • 9 slides
Recent Advances Aly Khawaja Outline STAR- CCM+: a complete simulation workflow Emphasis on

Recent Advances Aly Khawaja Outline STAR- CCM+: a complete simulation workflow Emphasis on

Meshing in STAR-CCM+: Recent Advances Aly Khawaja Outline STAR- CCM+: a complete simulation workflow Emphasis on pre-processing technology Recent advances in surface preparation and meshing Continue to enhance flexibility, control,

1.13k views • 30 slides
A Review of Fact-Checking, Fake News Detection and Argumentation Tariq Alhindi March 02, 2020

A Review of Fact-Checking, Fake News Detection and Argumentation Tariq Alhindi March 02, 2020

A Review of Fact-Checking, Fake News Detection and Argumentation Tariq Alhindi March 02, 2020 Outline 1. Introduction 2. Fact-Checking 3. Fake News Detection 4. Argumentation Outline 1. Introduction 2. Fact-Checking a. What

766 views • 72 slides
Ele lectio ions approach are you ready? Fact-checking for ed educators and fu future

Ele lectio ions approach are you ready? Fact-checking for ed educators and fu future

Ele lectio ions approach are you ready? Fact-checking for ed educators and fu future voters Mikko Salo, Kari Kivinen, Pirjo Sallinen & Valentina Uitto Brussels, 13.11.2018 Fight disinformation with media literacy event organized

298 views • 19 slides
Fact Checking Problem: Estimated that people will see more fake than real news by 2022

Fact Checking Problem: Estimated that people will see more fake than real news by 2022

Fact Checking Problem: Estimated that people will see more fake than real news by 2022 Data: Google Fact Check research dataset Input: Claim + Supporting evidence Output: True / Partly True / Not True Supporting Evidence

150 views • 3 slides
Forecasting BASFs Custom Material Demand Using ABC/XYZ Analysis Team 7 Travis Greene,

Forecasting BASFs Custom Material Demand Using ABC/XYZ Analysis Team 7 Travis Greene,

Forecasting BASFs Custom Material Demand Using ABC/XYZ Analysis Team 7 Travis Greene, Patrizia Mach, Triguna Ashin Wijaya , Li-Cheng Pan BASF - chemical company Zur Angabe der Klassifizierung (VERTRAULICH etc.) bitte dieses Textfeld im

327 views • 10 slides
Economics 2 Professor Christina Romer Spring 2018 Professor David Romer LECTURE 4 EXTENSIONS

Economics 2 Professor Christina Romer Spring 2018 Professor David Romer LECTURE 4 EXTENSIONS

Economics 2 Professor Christina Romer Spring 2018 Professor David Romer LECTURE 4 EXTENSIONS OF SUPPLY AND DEMAND ANALYSIS January 25, 2018 I. O VERVIEW II. R EVIEW OF THE S UPPLY AND D EMAND F RAMEWORK A. Two sides of the market (example:

609 views • 38 slides
Planning Tools and Techniques 1 Planning Stage Estimating Scheduling Resources

Planning Tools and Techniques 1 Planning Stage Estimating Scheduling Resources

Planning Stage & Planning Tools and Techniques 1 Planning Stage Estimating Scheduling Resources Dependencies Milestones Outputs 2 Scheduling Setting out all project activities & tasks logically so

778 views • 44 slides
Planning Stage 1 of 39 Project Life Cycle - Planning Stage Conceptualisation Planning

Planning Stage 1 of 39 Project Life Cycle - Planning Stage Conceptualisation Planning

Planning Stage 1 of 39 Project Life Cycle - Planning Stage Conceptualisation Planning Implementation Completion 2 Planning Stage Inputs Why plan? The planning process Objectives Work Breakdown Structure

709 views • 39 slides
Statistical Timing Analysis Statistical Timing Analysis g g y y Considering Spatially and

Statistical Timing Analysis Statistical Timing Analysis g g y y Considering Spatially and

Statistical Timing Analysis Statistical Timing Analysis g g y y Considering Spatially and Considering Spatially and Temporally Correlated Dynamic Temporally Correlated Dynamic Temporally Correlated Dynamic Temporally Correlated Dynamic

902 views • 28 slides
Outline Part I. Introduction Part II. ML for DI Part III. DI for ML Training data

Outline Part I. Introduction Part II. ML for DI Part III. DI for ML Training data

Outline Part I. Introduction Part II. ML for DI Part III. DI for ML Training data creation Data cleaning Part IV. Conclusions and research directions Successful ML requires Data Integration Large collections of manually

510 views • 15 slides
Measurement for Improvement Dr Jennifer Martin Dr Michael Carton Tuesday 9 th May 2017

Measurement for Improvement Dr Jennifer Martin Dr Michael Carton Tuesday 9 th May 2017

The Powers and Pitfalls of Measurement for Improvement Dr Jennifer Martin Dr Michael Carton Tuesday 9 th May 2017 Measurement for Improvement Team, QID Todays Presentation Framework for Improving Quality Measurement for Improvement

1.03k views • 38 slides
UQ, STAT2201, 2017, Lecture 9. Unit 10 Further Stats Overview 1 The Strength of Conditional

UQ, STAT2201, 2017, Lecture 9. Unit 10 Further Stats Overview 1 The Strength of Conditional

UQ, STAT2201, 2017, Lecture 9. Unit 10 Further Stats Overview 1 The Strength of Conditional Probability 27: Bayes Theorem 74.3: Bayesian Estimation of Parameters 2 Other Elementary Distributions 37: Geometric and Negative

456 views • 12 slides

More recommend