Data Quality Challenges ACM JDIQ EiC Open Knowledge Networks - PowerPoint PPT Presentation

Apr 03, 2023 •272 likes •482 views

Data Quality Challenges ACM JDIQ EiC Open Knowledge Networks (Biomedicine) Data Science for Finance (DSfin) Louiqa Raschid Smith School of Business Computer Science and UMIACS Technical Review of Data Quality Provenance

Data Quality Challenges ● ACM JDIQ EiC ● Open Knowledge Networks (Biomedicine) ● Data Science for Finance (DSfin) Louiqa Raschid Smith School of Business Computer Science and UMIACS
Technical Review of Data Quality ● Provenance Cleaning Annotation ● Data cleaning infrastructure and tools: ○ Robust first generation. ○ Big data and scalability. ○ Human-in-the-loop (HumInt). ● Process ○ Fitness to task. ○ Understanding workflows.
First Gen methodologies and products
Technical Review of Data Quality ● Provenance Cleaning Annotation ● Data cleaning infrastructure and tools: ○ Robust first generation. ○ Big data and scalability. ○ Human-in-the-loop (HumInt). ● Process ○ Fitness to task. ○ Understanding workflows.
Scenarios ● Lung cancer data (primary) generated by clinicians: ○ Patient entity identification in clinical notes, e.g., JM, J.M., etc. (cleaning) ○ Scale: barthel 4 ○ Stages: P0T0, Stage4, etc. (annotation) ● Analytics over (secondary) sources: ○ Drug induced liver injury (DILI): phenotype includes elevated levels of liver enzymes, etc. ○ (fitness to task; HumInt): There are many causes for elevated liver enzymes including transplants, some infants, etc.
Scenarios ● Privacy preserving data mining: ○ Entity linkage in the de-identified space. ○ Different entries contribute hashed identifiers but they may be missing a variety of fields. (provenance; fitness to task) ● iASiS SEMANTIC Data Cleaning / Annotation Pipeline ● Finding patterns in OKN: DILI Case Study ○ (Provenance; fitness to task; HumInt; Annotation.)
iASiS
DILI Case Study o Given a knowledge graph and a DILI phenotype (keywords) ... o Create profiles, e.g., [Phenotype | Drug | Gene | Pathway] o Rank the DRUG at most risk for DILI.
DILI Case Study
DILI Case Study
Tamr: Understanding Workflows
Tamr: Understanding Workflows
Tamr: Understanding Workflows
Lessons learned ● First generation tools work well. ● Next generation needs to focus on processes and workflows and HumInt. ● Scientists still spend huge amounts of time on cleaning. How can we fix this problem? ● Is Open Knowledge Networks a solution? ● An unexpected case study ...

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Challenges in Improving Information Quality NISS Data Quality Conference November 30

Challenges in Improving Information Quality NISS Data Quality Conference November 30 December 1 Ann Thornton National Director, Data Quality and Integrity Deloitte & Touche Perspective on Information Quality Inclusion within

705 views • 21 slides

The challenges of capturing clinically accurate data for electronic Clinical Quality Measures

The challenges of capturing clinically accurate data for electronic Clinical Quality Measures (eCQM) Joe Kunisch PhD, RN-BC, CPHQ Enterprise Director of Clinical Quality Informatics Memorial Hermann Hospital System Quality Patient Safety &

294 views • 14 slides

The Data Quality Management Challenges of Solvency II Massimiliano Neri, Associate Director,

The Data Quality Management Challenges of Solvency II Massimiliano Neri, Associate Director, Moodys analytics 2 Agenda 1. Introduction 2. Criteria to Assess Data Quality 3. Data Quality Systems and Procedures 4. Moodys Analytics Best

292 views • 26 slides

OPPORTUNITIES AND CHALLENGES FOR HEALTH DATA ORGANIZATIONS (USING AHRQ QUALITY INDICATORS)

ICD-10 IMPLEMENTATION: OPPORTUNITIES AND CHALLENGES FOR HEALTH DATA ORGANIZATIONS (USING AHRQ QUALITY INDICATORS) Patrick S. Romano, MD MPH Professor of Medicine and Pediatrics UC Davis School of Medicine Co-Editor in Chief, HSR AHRQ Quality

625 views • 25 slides

Data Collection and Aggregation Data Collection and Aggregation 1 Challenges: data Challenges:

Data Collection and Aggregation Data Collection and Aggregation 1 Challenges: data Challenges: data Data type: numerical sensor readings. Rich and massive data, spatially distributed and correlated. Data dynamics: data

713 views • 48 slides

DATA QUALITY AND DATA DATA QUALITY AND DATA PROGRAMMING PROGRAMMING "Data cleaning and

DATA QUALITY AND DATA DATA QUALITY AND DATA PROGRAMMING PROGRAMMING "Data cleaning and repairing account for about 60% of the work of data scientists." Christian Kaestner Required reading: Schelter, S., Lange, D., Schmidt, P.,

1.1k views • 86 slides

Census Data Quality Assurance 17 May 2010 Types of Quality Assurance (QA) Quality assurance of

Census Data Quality Assurance 17 May 2010 Types of Quality Assurance (QA) Quality assurance of captured and coded data Quality assurance of downstream processes (including data security and integrity) Quality assurance of population counts,

129 views • 12 slides

ESOs role as data provider: Strategies and Challenges ESOs mandate address the challenge:

ESOs role as data provider: Strategies and Challenges ESOs mandate address the challenge: Data Flow System provide quality content: Science Data Products future opportunities: ESO archive ASTERICS European Data Provider Forum, Heidelberg,

453 views • 16 slides

The challenges of f the data and how to obtain good quality Karsten Elmsted/Bent Reiter

The challenges of f the data and how to obtain good quality Karsten Elmsted/Bent Reiter EasyX Member of XBRL DK and XBRL SE About EasyX Worked with XBRL more than 15 years DK Gaap (2012), IFRS (2014), EBA (2014), EIOPA (2016)

241 views • 13 slides

After this session you should be able to: 1. Describe what is data quality and why it is

After this session you should be able to: 1. Describe what is data quality and why it is important. 2. Define methods of data quality check. 3. List reasons of poor data quality and explain how to address them. 4. Address process to improve

632 views • 42 slides

Data Quality Blueprint for Pentaho: Better Data Leads to Better Results Charles Gaddy Director

Data Quality Blueprint for Pentaho: Better Data Leads to Better Results Charles Gaddy Director Global Sales & Alliances, Melissa Data Agenda What is Data Quality, and What Role Does it Play? 6 Concepts of Data Quality Full Data

621 views • 26 slides

Quality data is needed to inform efforts to reduce disparities Have come a long way in the past

5/6/2014 Quality data is needed to inform efforts to reduce disparities Have come a long way in the past 10 years in identifying and exploring MCH issues experienced by the AI population Identified key challenges and barriers to better data,

367 views • 9 slides

R in Grenoble DATA CHALLENGES Magali Richard & Florent Chuffart Introduct ction Data

R in Grenoble DATA CHALLENGES Magali Richard & Florent Chuffart Introduct ction Data challenges in class Data challenges for scientists Tutorial CIT-SAB MEETING 2 What is a data challenge? Ellrott et al. Genome Biology (2019) 20:195

373 views • 21 slides

METROLOGY AND METROLOGY AND QUALITY ASSURANCE QUALITY ASSURANCE Challenges for the analytical

METROLOGY AND METROLOGY AND QUALITY ASSURANCE QUALITY ASSURANCE Challenges for the analytical chemist Challenges for the analytical chemist Ale Fajgelj Fajgelj Ale IAEA International Atomic Energy Agency Agencys Laboratories

1.3k views • 16 slides

Service Data Quality and Improvement Quality: When Session T31 99.7% Isnt Good May 17,

ASQ World Conference on Service Data Quality and Improvement Quality: When Session T31 99.7% Isnt Good May 17, 2016 Enough Jennifer Wilson Dodd Starbird Learning Objectives 1. Discuss the value of using quality data for process and

449 views • 22 slides

Childcare Challenges Finding and Providing Quality Childcare Overview Why does Quality

Childcare Challenges Finding and Providing Quality Childcare Overview Why does Quality Childcare Matter What does it look like? Challenges in providing quality childcare Why does Quality Child Care Matter? City

545 views • 25 slides

Benchmarking Adversarial Robustness on Image Classification Yinpeng Dong, Qi-An Fu, Xiao Yang,

Benchmarking Adversarial Robustness on Image Classification Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Zihao Xiao, Hang Su, Jun Zhu Dept. of Comp. Sci. and Tech., BNRist Center, Institute for AI, THBI Lab, Tsinghua University, Beijing,

417 views • 6 slides

S WEST COAST MARINE TOURISM COLLABORATION S WELCOME FROM WCMTC STEERING GROUP S Fiona McPhail,

WEST COAST MARINE TOURISM COLLABORATION SUMMIT 10 JANUARY 2019 SAMS OBAN S WEST COAST MARINE TOURISM COLLABORATION S WELCOME FROM WCMTC STEERING GROUP S Fiona McPhail, Tighnabruaich Sailing School, Argylls Secret Coast and AITC S Steven

1.55k views • 116 slides

Kindergarten to Sec Kindergarten to Second Grade ond Grade Task & T ask & Teacher Analy

Kindergarten to Sec Kindergarten to Second Grade ond Grade Task & T ask & Teacher Analy eacher Analysis sis Task: C ask: Constructiv onstructive c e con onversations about ersations about Grade L Grade Level: 1 el: 1

319 views • 5 slides

Kindergarten to Second Grade Interpreting Student Work Task: Constructive conversations about

Kindergarten to Second Grade Interpreting Student Work Task: Constructive conversations about Grade Level: 1 predicted animal adaptations Subject: Science/ELA Source: Understanding Language What do you do when something wants to eat you? by

70 views • 5 slides

14 Gb/s AC Coupled Receiver in 90 nm CMOS Masum Hossain & Tony Chan Carusone University of

14 Gb/s AC Coupled Receiver in 90 nm CMOS Masum Hossain & Tony Chan Carusone University of Toronto masum@eecg.utoronto.ca OUTLINE Chip-to-Chip link overview AC interconnects Link modelling ISI & sensitivity AC

665 views • 37 slides

Lecture 14: LPC speech synthesis and autocorrelation- based pitch tracking ECE 417, Multimedia

Lecture 14: LPC speech synthesis and autocorrelation- based pitch tracking ECE 417, Multimedia Signal Processing October 10, 2019 Outline The LPC-10 speech synthesis model Autocorrelation-based pitch tracking Inter-frame

768 views • 37 slides

Laser test progress in Prague Peter Kody, Zden k Doleal, Jan Bro, Peter Kvasni ka,

Charles University Prague Charles University Prague Institute of Particle and Nuclear Physics Institute of Particle and Nuclear Physics Laser test progress in Prague Peter Kody, Zden k Doleal, Jan Bro, Peter Kvasni ka, Pavel

497 views • 20 slides

BGC float navigation and parameters (CTS4 et CTS5-USEA) How we managed to deploy 206 BGC float

BGC float navigation and parameters (CTS4 et CTS5-USEA) How we managed to deploy 206 BGC float Antoine Poteau 1. CTS4/CTS5-USEA , BGC SENSORS 2. Mission 3. Acquisition 4. 206 deployements (25 differents PI) 1. CTS4/CTS5-USEA , BGC SENSORS

755 views • 19 slides