text mining on clinical data
play

Text Mining on Clinical Data Robert McHardy Outline Motivation - PowerPoint PPT Presentation

Dec 15, 2022 •345 likes •1.18k views

Institut fr Maschinelle Sprachverarbeitung Text Mining on Clinical Data Robert McHardy Outline Motivation Medical Entity Recognition Anonymization of Medical Reports Knowledge-based Biomedical Word Sense Disambiguation

  1. Institut für Maschinelle Sprachverarbeitung Text Mining on Clinical Data Robert McHardy

  2. Outline • Motivation • Medical Entity Recognition • Anonymization of Medical Reports • Knowledge-based Biomedical Word Sense Disambiguation • Extraction of Potential Adverse Drug Events • Resources Universität Stuttgart 5.12.2017 2

  3. Motivation — Different Users Universität Stuttgart 5.12.2017 3

  4. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not Universität Stuttgart 5.12.2017 4

  5. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible Universität Stuttgart 5.12.2017 4

  6. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible • We don‘t want to suffer from unsafe drugs Universität Stuttgart 5.12.2017 4

  7. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible • We don‘t want to suffer from unsafe drugs • Researchers want to use the data Universität Stuttgart 5.12.2017 4

  8. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible • We don‘t want to suffer from unsafe drugs • Researchers want to use the data • It has to be anonymized Universität Stuttgart 5.12.2017 4

  9. Motivation — PubMed, again! Universität Stuttgart 5.12.2017 5

  10. Unified Medical Language System Metathesaurus (UMLS) Universität Stuttgart 5.12.2017 6

  11. Medical Entity Recognition — Overview • Abacha and Zweigenbaum: Consists of two parts • Detecting phrases referring to medical entities • Assigning semantic categories to the found entities Universität Stuttgart 5.12.2017 7

  12. Medical Entity Recognition — Overview Universität Stuttgart 5.12.2017 8

  13. Medical Entity Recognition — Overview Type 1 diabetes T1D Diabetes type 1 IDDM Juvenile diabetes Universität Stuttgart 5.12.2017 8

  14. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] Universität Stuttgart 5.12.2017 9

  15. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Many tools for NP chunking available Universität Stuttgart 5.12.2017 9

  16. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Many tools for NP chunking available • Maximum recall is desired Universität Stuttgart 5.12.2017 9

  17. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Many tools for NP chunking available • Maximum recall is desired • Open-domain tools like IMS‘ TreeTagger are suitable Universität Stuttgart 5.12.2017 9

  18. Medical Entity Recognition — MetaMap and the UMLS • MetaMap is a tool which maps noun phrases in raw text to UMLS concepts • This is done according to a matching score Universität Stuttgart 5.12.2017 10

  19. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap Universität Stuttgart 5.12.2017 11

  20. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools Universität Stuttgart 5.12.2017 11

  21. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs Universität Stuttgart 5.12.2017 11

  22. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs • Some ambiguity is left Universität Stuttgart 5.12.2017 11

  23. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs • Some ambiguity is left • UMLS can provide several concepts for a term Universität Stuttgart 5.12.2017 11

  24. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs • Some ambiguity is left • UMLS can provide several concepts for a term • and several semantic categories for a concept Universität Stuttgart 5.12.2017 11

  25. Medical Entity Recognition — MetaMap and the UMLS Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] Cold temperature Common cold Cold ( term) Cold storage ( term) Cold storage Chronic obstructive lung disease Universität Stuttgart 5.12.2017 12

  26. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking Universität Stuttgart 5.12.2017 13

  27. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list Universität Stuttgart 5.12.2017 13

  28. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list • Search in specialized lists for candidate terms Universität Stuttgart 5.12.2017 13

  29. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list • Search in specialized lists for candidate terms • Annotate entities with MetaMap Universität Stuttgart 5.12.2017 13

  30. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list • Search in specialized lists for candidate terms • Annotate entities with MetaMap • Filter frequent errors and too broad semantic types Universität Stuttgart 5.12.2017 13

  31. Medical Entity Recognition — MetaMap+ • Voting mechanism to disambiguate semantic categories Universität Stuttgart 5.12.2017 14

  32. Medical Entity Recognition — Support Vector Machines (SVMs) • Word level features: • words of the NP • number of words of the NP • window of words around the NP • Orthographical features: • first letter capitalized • all letters upper-/lowercase • contains abbreviation(s) • POS tags Universität Stuttgart 5.12.2017 15

  33. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O Universität Stuttgart 5.12.2017 16

  34. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O • B-x: Begin of a phrase of class x Universität Stuttgart 5.12.2017 16

  35. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O • B-x: Begin of a phrase of class x • I-x: Intermediate part of a phrase of class x Universität Stuttgart 5.12.2017 16

  36. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O • B-x: Begin of a phrase of class x • I-x: Intermediate part of a phrase of class x • O: Outside entities Universität Stuttgart 5.12.2017 16

  37. Medical Entity Recognition — BIO-CRFs • Word level features: • The word itself • Window of words • Lemmas • Orthographical features: • Upper/lowercase • contains a digit • pre- and suffixes • POS tags • (Semantic category of word (provided by MetaMap+)) Universität Stuttgart 5.12.2017 17

  38. Medical Entity Recognition — Evaluation • Corpus contains discharge summaries and progress notes • De-identified and annotated by hand • Entities: Problem, Treatment and Test • Overall 76,665 sentences Universität Stuttgart 5.12.2017 18

  39. Medical Entity Recognition — Evaluation Setting Precision Recall F-Score MetaMap 15.52 16.10 15.80 MetaMap+ 48.68 56.46 52.28 SVM 43.65 47.16 45.33 BIO-CRF 70.15 83.31 76.17 BIO-CRF-Hybrid 72.18 83.78 77.55 Universität Stuttgart 20.01.2016 19

  40. Anonymization of Medical Reports Universität Stuttgart 20.01.2016 20

  41. Anonymization of Medical Reports — What is anonymization? • De-Identification Universität Stuttgart 5.12.2017 21

  42. Anonymization of Medical Reports — What is anonymization? • De-Identification • Completely remove all personal health information Universität Stuttgart 5.12.2017 21

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies

904 views • 64 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com Tutorial on Machine Learning with R The Melbourne Data Science Week 2017 1 June 2017 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies .

733 views • 57 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011 Machine Learning & Computational Biology Research Group MPIs Tbingen .: Data Mining in Bioinformatics, Page 1 What is text mining?

813 views • 31 slides
2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118 / 179 Text Mining Goals To learn key problems and techniques in the mining one of the most common types of data To learn how to represent text

738 views • 59 slides
Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Motivation for Text Mining Motivation for Text Mining Approximately 90% of the Worlds data is held in unstructured formats Text Mining Text Mining Web pages Emails Technical documents Corporate documents Books

456 views • 9 slides
CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun yzsun@cs.ucla.edu December 4, 2017 Methods to be Learnt Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for Text

418 views • 30 slides
Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao

Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao

Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao http://www.RDataMining.com Short Course on R and Data Mining University of Canberra 7 October 2016 1 Chapter 10: Text Mining, in R and Data Mining:

798 views • 40 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10 Bioinformatics Group MPIs Tbingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 What is text mining? Definition Text mining is the use

603 views • 31 slides
Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early stage demen1a: the SAMS framework Christopher Bull, Dommy Asfiandy, Ann Gledson, Joseph

284 views • 14 slides
Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow) & David Robinson (DataCamp) https://www.tidytextmining.com/ R for Data Science: Garrett Grolemund & Hadley Wickham http://r4ds.had.co.nz/

866 views • 15 slides
Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire Beatrice Alex balex@inf.ed.ac.uk Digital scholarship: day of ideas 2, Edinburgh, 02/05/2013 OVERVIEW What is text mining? Text Mining in digital

484 views • 27 slides
CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu

CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu

CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu February 17, 2016 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Decision Tree;

337 views • 17 slides
Data Mining in the Humanities Text Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar,

Data Mining in the Humanities Text Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar,

Data Mining in the Humanities Text Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar, University of Edinburgh November 19th 2013 Monday, 25 November 2013 MINING WHAT DATA? Electronic text or things that can be turned into it. Born

911 views • 51 slides
Data Mining 2020 Text Classification Naive Bayes Ad Feelders Universiteit Utrecht Ad Feelders

Data Mining 2020 Text Classification Naive Bayes Ad Feelders Universiteit Utrecht Ad Feelders

Data Mining 2020 Text Classification Naive Bayes Ad Feelders Universiteit Utrecht Ad Feelders ( Universiteit Utrecht ) Data Mining 1 / 49 Text Mining Text Mining is data mining applied to text data. Often uses well-known data mining

597 views • 49 slides
MINING Text Data: Nave Bayes Instructor: Yizhou Sun yzsun@cs.ucla.edu December 7, 2017

MINING Text Data: Nave Bayes Instructor: Yizhou Sun yzsun@cs.ucla.edu December 7, 2017

CS145: INTRODUCTION TO DATA MINING Text Data: Nave Bayes Instructor: Yizhou Sun yzsun@cs.ucla.edu December 7, 2017 Methods to be Learnt Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for Text

903 views • 26 slides
Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt

Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt

Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt February 25 to March 10 Bioinformatics Group MPIs Tbingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 Why compare sequences?

786 views • 25 slides
Instant classroom response system on mobile Instant classroom response system on mobile devices

Instant classroom response system on mobile Instant classroom response system on mobile devices

Instant classroom response system on mobile Instant classroom response system on mobile devices uReply user experiences devices uReply user experiences Speakers: Prof. LAM L. C. Paul, Associate Professor of Centre for Learning Enhancement

776 views • 19 slides
TRANSCRIPT About Dr. David Harrison After completing his medical studies, David founded the

TRANSCRIPT About Dr. David Harrison After completing his medical studies, David founded the

TRANSCRIPT About Dr. David Harrison After completing his medical studies, David founded the Health Systems Trust, a non- government organisation supporting health services development in South Africa. He started up the South African Health

477 views • 15 slides
As Assi sisted R Repr eproductive T e Tec echnology ( (AR ART) in n Eur urope 201 e

As Assi sisted R Repr eproductive T e Tec echnology ( (AR ART) in n Eur urope 201 e

As Assi sisted R Repr eproductive T e Tec echnology ( (AR ART) in n Eur urope 201 e 2016 and d d dev evel elopm pmen ent o of a strategy o of vigilance Prelim limin inary r result lts g generated f from European r regis

605 views • 36 slides
What Is Known About the Human Health Effects of Neonicotinoid Pesticides? Melissa Perry, ScD,

What Is Known About the Human Health Effects of Neonicotinoid Pesticides? Melissa Perry, ScD,

What Is Known About the Human Health Effects of Neonicotinoid Pesticides? Melissa Perry, ScD, MHS Department of Environmental and Occupational Health Milken Institute School of Public Health The George Washington University What are neonics?

764 views • 33 slides
Disclaimer: The material herein is developed under NSF-NUE (Nanotechnology Undergraduate

Disclaimer: The material herein is developed under NSF-NUE (Nanotechnology Undergraduate

NanoTRA -Texas Regional Alliance to Foster Nanotechnology Environment, Health, and Safety http://nsf-nue-nanotra.engineering.txstate.edu/ Disclaimer: The material herein is developed under NSF-NUE (Nanotechnology Undergraduate Education) award

785 views • 41 slides
Luis Mendez Hala Altunji Johnpaul Golinski Sol Ezra 5/4/2011 Executive Summary Problem:

Luis Mendez Hala Altunji Johnpaul Golinski Sol Ezra 5/4/2011 Executive Summary Problem:

Luis Mendez Hala Altunji Johnpaul Golinski Sol Ezra 5/4/2011 Executive Summary Problem: tissue engineering is limited to 2D scaffolds. Objective: improve tissue engineering by creating a useful, biocompatible 3D scaffold.

339 views • 21 slides
Fecal Microbiota Transplantation in C. diff. colitis Benefits and Limitations Gerhard Rogler,

Fecal Microbiota Transplantation in C. diff. colitis Benefits and Limitations Gerhard Rogler,

January 27th 2017, 8th Gastro Foundation Weekend for Fellows; Spier Hotel & Conference Centre, Stellenbosch Fecal Microbiota Transplantation in C. diff. colitis Benefits and Limitations Gerhard Rogler, Department of Gastroenterology and

499 views • 18 slides
FRIENDS OF CHILDREN WITH CANCER - TANZANIA FRIENDS OF CHILDREN IN HOSPITALS F RIENDS OF C HILDREN

FRIENDS OF CHILDREN WITH CANCER - TANZANIA FRIENDS OF CHILDREN IN HOSPITALS F RIENDS OF C HILDREN

FRIENDS OF CHILDREN WITH CANCER - TANZANIA FRIENDS OF CHILDREN IN HOSPITALS F RIENDS OF C HILDREN WITH C ANCER - O VERVIEW It is a Network of friends supporting the needs of the families affected by NCDs in Tanzania Key Objective ! To raise

497 views • 21 slides

More recommend