the impact of distributional metrics in the quality of
play

The Impact of Distributional Metrics in the Quality of Relational - PowerPoint PPT Presentation

Apr 10, 2023 •45 likes •665 views

The Impact of Distributional Metrics in the Quality of Relational Triples calo Oliveira 1 , Paulo Gomes Hernani Costa, Hugo Gon hpcosta@student.dei.uc.pt, { hroliv,pgomes } @dei.uc.pt Cognitive & Media Systems Group CISUC, University of

  1. The Impact of Distributional Metrics in the Quality of Relational Triples calo Oliveira 1 , Paulo Gomes Hernani Costa, Hugo Gon¸ hpcosta@student.dei.uc.pt, { hroliv,pgomes } @dei.uc.pt Cognitive & Media Systems Group CISUC, University of Coimbra Lisbon, August 16, 2010 1supported by FCT scholarship grant SFRH/BD/44955/2008 Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 1 / 21

  2. Outline Introduction 1 Information Extraction Information Retrieval Research Goals Approach 2 Experimentation 3 Set-up Metrics adaptation Results Additional experimentation Concluding remarks 4 Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 2 / 21

  3. Introduction Introduction Knowledge bases (eg. WordNet) are useful resources for NLP Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 3 / 21

  4. Introduction Introduction Knowledge bases (eg. WordNet) are useful resources for NLP Their creation and maintenance involves intensive human effort Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 3 / 21

  5. Introduction Introduction Knowledge bases (eg. WordNet) are useful resources for NLP Their creation and maintenance involves intensive human effort Automatic creation/enrichment from textual resources is an alternative Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 3 / 21

  6. Introduction Introduction Knowledge bases (eg. WordNet) are useful resources for NLP Their creation and maintenance involves intensive human effort Automatic creation/enrichment from textual resources is an alternative ▶ Higher coverage, easier update, but... Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 3 / 21

  7. Introduction Introduction Knowledge bases (eg. WordNet) are useful resources for NLP Their creation and maintenance involves intensive human effort Automatic creation/enrichment from textual resources is an alternative ▶ Higher coverage, easier update, but... ▶ Precision is lower Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 3 / 21

  8. Introduction Introduction Knowledge bases (eg. WordNet) are useful resources for NLP Their creation and maintenance involves intensive human effort Automatic creation/enrichment from textual resources is an alternative ▶ Higher coverage, easier update, but... ▶ Precision is lower ▶ Evaluation requires once again intensive human labour! Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 3 / 21

  9. Introduction Information Extraction Information extraction (IE) Automatic extraction of structured information from natural language. Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 4 / 21

  10. Introduction Information Extraction Information extraction (IE) Automatic extraction of structured information from natural language. “Car is a vehicle with 4 wheels and an engine, used for carrying a small number of passengers.” Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 4 / 21

  11. Introduction Information Extraction Information extraction (IE) Automatic extraction of structured information from natural language. “Car is a vehicle with 4 wheels and an engine, used for carrying a small number of passengers.” ▶ vehicle HYPERNYM OF car Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 4 / 21

  12. Introduction Information Extraction Information extraction (IE) Automatic extraction of structured information from natural language. “Car is a vehicle with 4 wheels and an engine, used for carrying a small number of passengers.” ▶ vehicle HYPERNYM OF car ▶ wheel PART OF car ▶ engine PART OF car Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 4 / 21

  13. Introduction Information Extraction Information extraction (IE) Automatic extraction of structured information from natural language. “Car is a vehicle with 4 wheels and an engine, used for carrying a small number of passengers.” ▶ vehicle HYPERNYM OF car ▶ wheel PART OF car ▶ engine PART OF car ▶ carrying people PURPOSE OF car Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 4 / 21

  14. Introduction Information Retrieval Information retrieval (IR) Locating specific information in natural language resouces. Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 5 / 21

  15. Introduction Information Retrieval Information retrieval (IR) Locating specific information in natural language resouces. Approaches based on the occurrence of words in documents. Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 5 / 21

  16. Introduction Information Retrieval Information retrieval (IR) Locating specific information in natural language resouces. Approaches based on the occurrence of words in documents. Distributional similarity metrics ▶ Cocitation (Small (1973)) ▶ LSA (Deerwester et al. (1990)) ▶ Lin’s (Lin (1998)) ▶ PMI-IR (Turney (2001)) ▶ 휎 (Kozima and Furugori (1993)) ▶ ... Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 5 / 21

  17. Introduction Research Goals Goals 1 Use IR metrics to improve IE precision ▶ Adapt distributional metrics to determine words similarity Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 6 / 21

  18. Introduction Research Goals Goals 1 Use IR metrics to improve IE precision ▶ Adapt distributional metrics to determine words similarity ▶ Wandmacher et al. (2007) and Cederberg and Widdows (2003) used LSA to weight hypernymy triples Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 6 / 21

  19. Introduction Research Goals Goals 1 Use IR metrics to improve IE precision ▶ Adapt distributional metrics to determine words similarity ▶ Wandmacher et al. (2007) and Cederberg and Widdows (2003) used LSA to weight hypernymy triples ▶ What about other semantic relations? Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 6 / 21

  20. Introduction Research Goals Goals 1 Use IR metrics to improve IE precision ▶ Adapt distributional metrics to determine words similarity ▶ Wandmacher et al. (2007) and Cederberg and Widdows (2003) used LSA to weight hypernymy triples ▶ What about other semantic relations? ▶ What metrics should be used? Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 6 / 21

  21. Introduction Research Goals Goals 1 Use IR metrics to improve IE precision ▶ Adapt distributional metrics to determine words similarity ▶ Wandmacher et al. (2007) and Cederberg and Widdows (2003) used LSA to weight hypernymy triples ▶ What about other semantic relations? ▶ What metrics should be used? ▶ New combined metrics? Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 6 / 21

  22. Introduction Research Goals Goals 1 Use IR metrics to improve IE precision ▶ Adapt distributional metrics to determine words similarity ▶ Wandmacher et al. (2007) and Cederberg and Widdows (2003) used LSA to weight hypernymy triples ▶ What about other semantic relations? ▶ What metrics should be used? ▶ New combined metrics? 2 Help manual evaluation Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 6 / 21

  23. Approach IE system Grammars Extraction of Corpus relational triples Removal of triples with stopwords Lemmatisation Additional Metrics extraction of triples application Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 7 / 21

  24. Experimentation Set-up Experimentation set-up ublico 2 corpus (annotated version) CETEMP´ ▶ 28,000 documents ▶ 30,100 unique context words (nouns, verbs and adjectives) ▶ term-document matrix 2 http://www.linguateca.pt/cetempublico/ Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 8 / 21

  25. Experimentation Set-up Experimentation set-up ublico 2 corpus (annotated version) CETEMP´ ▶ 28,000 documents ▶ 30,100 unique context words (nouns, verbs and adjectives) ▶ term-document matrix Triples obtained ▶ Extracted: 20,308 ▶ Discarded: 5,844 ▶ Inferred: 2,492 ▶ Final triple set: 16,956 2 http://www.linguateca.pt/cetempublico/ Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 8 / 21

  26. Experimentation Metrics adaptation Similarity between two documents For instance, Cocitation: First presented as a similarity metric between scientific papers (Small (1973)) Cocitation ( d i , d j ) = P ( d i ∩ d j ) (1) P ( d i ∪ d j ) Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 9 / 21

  27. Experimentation Metrics adaptation Similarity between two documents For instance, Cocitation: First presented as a similarity metric between scientific papers (Small (1973)) Cocitation ( d i , d j ) = P ( d i ∩ d j ) (1) P ( d i ∪ d j ) ▶ d i , d j represent two documents Costa, Gon¸ calo Oliveira & Gomes (CISUC) LaTeCH 2010 Lisbon, August 16, 2010 9 / 21

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

THE DISTRIBUTIONAL IMPACT OF POPULATION AGEING Affording Our Future Conference 10-11 December,

THE DISTRIBUTIONAL IMPACT OF POPULATION AGEING Affording Our Future Conference 10-11 December,

THE DISTRIBUTIONAL IMPACT OF POPULATION AGEING Affording Our Future Conference 10-11 December, 2012 Omar A. Aziz, Chris Ball, John Creedy and Jesse Unger LIVING STANDARDS: THE HEART OF OUR POLICY ADVICE Assess the impact of policy across key

509 views • 28 slides
Software Quality Metrics What does software quality mean? and How is it measured? Factors in

Software Quality Metrics What does software quality mean? and How is it measured? Factors in

Software Quality Metrics What does software quality mean? and How is it measured? Factors in Software Quality Factor Criteria Metric Maintainability Simplicity Modularity Conciseness Nesting Level Modules that Percentage of exceed size

256 views • 8 slides
PANEL 4: METRICS OF PROGRESS AND MEASURES OF IMPACT INCLUDING COST- EFFECTIVENESS, OUTCOMES TO

PANEL 4: METRICS OF PROGRESS AND MEASURES OF IMPACT INCLUDING COST- EFFECTIVENESS, OUTCOMES TO

PANEL 4: METRICS OF PROGRESS AND MEASURES OF IMPACT INCLUDING COST- EFFECTIVENESS, OUTCOMES TO VALUE TO PAYERS, INFLUENCING QUALITY OF CARE THROUGH LEARNING HEALTHCARE SYSTEMS Genomic Medicine VIII This meeting will help NHGRI and its Genomic

542 views • 28 slides
The Distributional Impact of Tax-Benefit Systems in Six African Countries Katrin Gasior, Chrysa

The Distributional Impact of Tax-Benefit Systems in Six African Countries Katrin Gasior, Chrysa

The Distributional Impact of Tax-Benefit Systems in Six African Countries Katrin Gasior, Chrysa Leventi, Michael Noble, Gemma Wright & Helen Barnes WIDER Development Conference, Helsinki, 13 th September 2018 Background Taxation and

356 views • 17 slides
Project Management Metrics SE 350 Software Processes & Product Quality Project Management

Project Management Metrics SE 350 Software Processes & Product Quality Project Management

Project Management Metrics SE 350 Software Processes & Product Quality Project Management Metrics Cycletime Productivity Staffing Requirements volatility Reuse metrics Activity progress measurement Estimation

325 views • 21 slides
Medical Device Quality Metrics FDA/Xavier University Initiative MDIC Case for Quality Forum June

Medical Device Quality Metrics FDA/Xavier University Initiative MDIC Case for Quality Forum June

Medical Device Quality Metrics FDA/Xavier University Initiative MDIC Case for Quality Forum June 28, 2016 Purpose and Outcome Purpose: To provide a system of metrics that Supports the Case for Quality Spans the Total Product Lifecycle

635 views • 31 slides
Fundamentals SE 350 Software Process & Product Quality Lecture Objectives Provide some

Fundamentals SE 350 Software Process & Product Quality Lecture Objectives Provide some

Measurement and Metrics Fundamentals SE 350 Software Process & Product Quality Lecture Objectives Provide some basic concepts of metrics Quality attribute metrics and measurements Reliability, validity, error Correlation

896 views • 37 slides
Distributional Semantics The unsupervised modeling of meaning on a large scale Tim Van de Cruys

Distributional Semantics The unsupervised modeling of meaning on a large scale Tim Van de Cruys

equipe melodi Distributional Semantics The unsupervised modeling of meaning on a large scale Tim Van de Cruys IRIT, Toulouse Tuesday 17 November 2015 Distributional similarity distributional hypothesis [Harris 1954] 2/52 Distributional

1.01k views • 88 slides
Quality Metrics Work Group November 16, 2015 1 Agenda Introductions SIM and Care

Quality Metrics Work Group November 16, 2015 1 Agenda Introductions SIM and Care

SIM Quality Metrics Work Group November 16, 2015 1 Agenda Introductions SIM and Care Coordination Overview Overview of Quality Metrics Work Group Discussion on Quality Performance Reporting Initiatives SIM to Support Short and

409 views • 8 slides
NAHDO Data Quality Forum Cross State Metrics to Assess Data Quality September 2019 Identify

NAHDO Data Quality Forum Cross State Metrics to Assess Data Quality September 2019 Identify

NAHDO Data Quality Forum Cross State Metrics to Assess Data Quality September 2019 Identify cross-state metrics to compare data quality benchmarks Use NAHDOs Data Quality Forum to gather input and consensus on an agreed upon set of

275 views • 6 slides
1 QUALITY ASSURANCE - PROCESS QUALITY METRICS PREREQUISITES Top management commitment

1 QUALITY ASSURANCE - PROCESS QUALITY METRICS PREREQUISITES Top management commitment

SOFTWARE ENGINEERING SOFTWARE QUALITY - SOFTWARE QUALITY QUALITY COMPONENTS Today we talk about software process quality and Objective quality component: properties that can certification be measured or approximated objectively

237 views • 5 slides
Impact of particles on photolysis rates and Results Impact on photolysis air quality rates

Impact of particles on photolysis rates and Results Impact on photolysis air quality rates

Introduction Model description Impact of particles on photolysis rates and Results Impact on photolysis air quality rates Impact on concentrations Impact on air quality monitoring Elsa Real a and Karine Sartelet a Conclusions 3 juin

521 views • 20 slides
Metrics & Scoring Committee: Recommendations to Health Plan Quality Metrics Committee, 2019

Metrics & Scoring Committee: Recommendations to Health Plan Quality Metrics Committee, 2019

Metrics & Scoring Committee: Recommendations to Health Plan Quality Metrics Committee, 2019 Measure Set Will Brake, Committee Chair, AllCare Health Anna Jimenez, MD, Committee Vice-Chair, Family Care CCO November 9, 2017 HEALTH POLICY

191 views • 15 slides
Poverty and distributional impact of Covid-19 Crisis in Indonesia Arief Anshory Yusuf SDGs

Poverty and distributional impact of Covid-19 Crisis in Indonesia Arief Anshory Yusuf SDGs

Poverty and distributional impact of Covid-19 Crisis in Indonesia Arief Anshory Yusuf SDGs Center, Universitas Padjadjaran http://sdgcenter.unpad.ac.id Presented at How is COVID-19 changing development? WIDER Webinar Series, UNU/WIDER, 12

622 views • 21 slides
Can We Predict Scientific Impact with Social Media? A Comparison with Traditional Metrics of

Can We Predict Scientific Impact with Social Media? A Comparison with Traditional Metrics of

Can We Predict Scientific Impact with Social Media? A Comparison with Traditional Metrics of Scientific Impact Denis Helic Knowledge Technologies Institute, Graz University of Technology December 04, 2014 Helic (KTI) Scientific Impact

799 views • 52 slides
Distributional Compositionality Intro to Distributional Semantics Raffaella Bernardi University

Distributional Compositionality Intro to Distributional Semantics Raffaella Bernardi University

Distributional Compositionality Intro to Distributional Semantics Raffaella Bernardi University of Trento February 14, 2012 Acknowledgments Credits: Some of the slides of today lecture are based on earlier DS courses taught by Marco Baroni,

538 views • 49 slides
Multi-agent constrained optimization of a strongly convex function Necdet Serhat Aybat

Multi-agent constrained optimization of a strongly convex function Necdet Serhat Aybat

Multi-agent constrained optimization of a strongly convex function Necdet Serhat Aybat Industrial & Manufacturing Engineering Penn State University Joint work with Erfan Yazdandoost Hamedani Research supported by NSF CMMI-1635106 and ARO

1k views • 77 slides
INDUSTRIES INC. CORPORATE PRESENTATION Financial Results for Fiscal 2018 and Second Quarter

INDUSTRIES INC. CORPORATE PRESENTATION Financial Results for Fiscal 2018 and Second Quarter

LASSONDE INDUSTRIES INC. CORPORATE PRESENTATION Financial Results for Fiscal 2018 and Second Quarter 2019 August 9, 2019 FORWARD-LOOKING STATEMENTS Caution regarding forward-looking statements Certain statements made in this presentation,

510 views • 39 slides
Technical Considerations for In-Beam Gamma-Ray Experiments at the RIBF P . Doornenbal

Technical Considerations for In-Beam Gamma-Ray Experiments at the RIBF P . Doornenbal

Technical Considerations for In-Beam Gamma-Ray Experiments at the RIBF P . Doornenbal Outline Accelerators Accelerators BigRIPS BigRIPS F8 Area Secondary and total beam rates Summary PID

473 views • 25 slides
Managem ent of paediatric severe sepsis-a brief overview Presented by: Radu Botgros, MD EMA

Managem ent of paediatric severe sepsis-a brief overview Presented by: Radu Botgros, MD EMA

Managem ent of paediatric severe sepsis-a brief overview Presented by: Radu Botgros, MD EMA Paediatric Coordinator, Infectious Diseases Specialist An agency of the European Union A.Control the source of infection Antibiotics Broad

349 views • 12 slides
Contents Company Introduction 01 DMDC Introduction 02 Quality Control 03 Honor &

Contents Company Introduction 01 DMDC Introduction 02 Quality Control 03 Honor &

Contents Company Introduction 01 DMDC Introduction 02 Quality Control 03 Honor & Certifications 04 05 Sales Network Chongqing Changfeng Biotechnology Co.Ltd Chongqing Chemical

297 views • 12 slides
Everywhere. First Aid products for work places Best function and quality on market

Everywhere. First Aid products for work places Best function and quality on market

First Aid for everyone. Everywhere. First Aid products for work places Best function and quality on market Self-explanatory products for laymen Range for all needs in emergency situations Typical target group: Manufacturing

723 views • 21 slides
Check Requests Welcome! We will begin promptly at 9 a.m. Make sure your first and last

Check Requests Welcome! We will begin promptly at 9 a.m. Make sure your first and last

Check Requests Welcome! We will begin promptly at 9 a.m. Make sure your first and last name displays in the Participant list so we can mark your attendance. Training materials: https://go.iu.edu/buyhandouts BUY.IU

1.1k views • 80 slides
Non-Catalog Orders Welcome! We will begin at 1:30 p.m. Make sure your first and last

Non-Catalog Orders Welcome! We will begin at 1:30 p.m. Make sure your first and last

Non-Catalog Orders Welcome! We will begin at 1:30 p.m. Make sure your first and last name displays in the Participant list so we can mark your attendance. Training materials: https://go.iu.edu/buyhandouts BUY.IU training

709 views • 68 slides

More recommend