Automatic generation of MedDRA terms groupings using an ontology - - PowerPoint PPT Presentation
Automatic generation of MedDRA terms groupings using an ontology Gunnar DECLERCK a , Cdric BOUSQUET a,b and Marie Christine JAULENT a a INSERM, UMRS 872 EQ20, Universit Paris Descartes, France. b Department of Public Health, CHU University
a INSERM, UMRS 872 EQ20, Université Paris Descartes, France. b Department of Public Health, CHU University of Saint Etienne, France.
MIE 2012, August 27th, Pisa
MedDRA (Medical Dictionary for Drug Regulatory Activities) : standard terminology used to code adverse drug reactions (ADRs) in safety reports for postmarketing drug surveillance. Used for Signal detection :
for the US, WHO Vigibase for Europe)
terms defining a unique medical condition) and drugs (a “signal”)
2 MedDRA hierarchical structure
WP Work Package title 1 Project management and administration 2 Framework for pharmacoepidemiological studies 3 Methods for signal detection 4 New tools for data collection from consumers 5 Benefit-risk integration and representation 6 Validation studies involving an Extended Audience 7 Training and communication
Main goal: To develop and evaluate semantic driven methods for grouping MedDRA terms to improve signal detection performances. Hypothesis: Signal detection is improved when algorithms use groupings of MedDRA terms referring to the same ADR condition (rather than one single term). Need a way to assist human experts to build MedDRA groupings (currently made manually) Method: Building an ADR ontology (OntoADR) providing MedDRA terms with formal machine- processable defintions to support automatic MedDRA terms grouping procedures (OWL queries selecting terms on the basis of their semantic properties).
3
IMI (Innovative Medicines Initiative) project (2009- 2014).
4
MedDRA terminology enriched with Snomed-CT concepts formal definitions 34994 concepts (20856 MedDRA 13.0 terms / others from Snomed-CT) 26 Snomed-CT relations used to express medical meaning of MedDRA concepts
HASFINDINGSITE specifies the body site affected by a condition HASOCCURRENCE refers to the specific period of life during which a condition first occurs …
Hierarchical relations from SOC level to PT (Preferred terms) level are converted to subsomption (subclass_of) relations LLT (low level terms) are integrated as annotation labels of PT concepts
PT level SOC level HLT level HLGT level
5
When it is possible, MedDRA concepts are mapped with Snomed-CT concepts via UMLS metathesaurus Semantic information describing Snomed-CT concepts used to build the formal definition of MedDRA concepts. When mapping impossible, formal definition made manually (via collaboration between knowledge engineers and medical experts)
Formal definition of « Dilatation intrahepatic duct congenital » Meddra PT concept after Snomed‐CT mapping (mapped with “Congenital dilatation of lobar intrahepatic bile duct”)
6
SOC HLPT HLT PT
defined using (i) a direct mapping with a Snomed‐CT concept (UMLS or other mappings methods) or (ii) a handmade definition.
terms used in the FDA database (calculated for the period 2004‐2010: 11 millions reports).
Mapping process
Medical experts: Validation of mappings Manual enrichment of OntoADR = MEDDRA TERMS + FORMAL DEFINITIONS
7
Thanks to OntoADR, OWL queries can be built to select the MedDRA PTs whose formal definition fits some definitional criteria. Example : Query to catch MedDRA terms related to “Upper gastrointestinal bleeding” Will select from MedDRA hierarchy all PTs matching those two properties:
Duodenal ulcer haemorrhage Gastric haemorrhage Mallory-Weiss syndrome Peptic ulcer haemorrhage etc. hasAssociatedMorphology some ‘Hemorrhage’ AND hasFindingSite some ‘Upper gastrointestinal tract structure’
8
Through the subsomption mechanism, MedDRA terms referring to hemorrhages of parts of the Upper gastrointestinal tract are also selected.
9
Focus on 13 ADR safety topics identified by Trifirò et al (2009) as first importance pharmacovigilance targets (EU-ADR project). For each safety topic: i. Groupings
MedDRA PTs have been realized with OntoADR queries.
ST 1 Bullous eruptions ST 2 Acute renal failure ST 3 Anaphylactic shock ST 4 Rhabdomyolysis ST 5 Aplastic anaemia/pancytopenia ST 6 Neutropenia ST 7 Cardiac valve fibrosis ST 8 Extrapyramidal disorders ST 9 Confusional state ST 10 Thrombocytopenia ST 11 Upper gastrointestinal bleeding ST 12 Peripheral neuropathy ST 13 Maculo‐papular erythematous eruptions
ADR topics identified by Trifirò et al (2009) 10
Trifirò G, Pariente A, Coloma PM, Kors JA, Polimeni G, Miremont‐Salamé G, Catania MA, Salvo F, David A, Moore N, Caputi AP, Sturkenboom M, Molokhia M, Hippisley‐Cox J, Acedo CD, van der Lei J, Fourrier‐Reglat A. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Saf. 2009; 18(12):1176‐84.
ii. The content of those groupings has been evaluated by comparison with existing handmade MedDRA groupings of PTs targeting same or close conditions:
A. Original MedDRA hierarchy groupings (HLTs or HLGTs) B. SMQs (Standard Medical Queries): collections of MedDRA PTs developed manually by the MSS0 (Maintenance and Support Services Organization) to describe a common clinical condition
Type (for SMQ) Label Id Meddra Agranulocytosis 10001507 Autoimmune neutropenia 10055128 Cyclic neutropenia 10053176 Febrile neutropenia 10016288 Felty's syndrome 10016386 Granulocytopenia 10018687 Granulocytopenia neonatal 10018688 Idiopathic neutropenia 10051645 Infantile genetic agranulocytosis 10052210 Neutropenia 10029354 Neutropenia neonatal 10029358 Neutropenic colitis 10062959 Neutropenic infection 10059482 Neutropenic sepsis 10049151 Narrow Agranulocytosis 10001507 Narrow Band neutrophil count decreased 10057950 Narrow Band neutrophil percentage decreased 10059130 Narrow Cyclic neutropenia 10053176 Narrow Febrile neutropenia 10016288 Narrow Idiopathic neutropenia 10051645 Narrow Neutropenia 10029354 Narrow Neutropenic infection 10059482 Narrow Neutropenic sepsis 10049151 Narrow Neutrophil count decreased 10029366 Broad Myelocyte percentage decreased 10052227 Broad Neutropenia neonatal 10029358 Broad Neutrophil count abnormal 10061313 Broad Neutrophil percentage decreased 10052223
Neutropenia
broader (refers to a deficit in the number of all types of white blood cells)
enable comparison with query-based neutropenia grouping, only neutropenia relevant PTs were selected in the SMQ
HLT Neutropenias SMQ Leukopenia
number of neutrophils” (*), which are the most abundant type of white blood cells (leukocytes) in mammals.
(*) http://www.nlm.nih.gov/medlineplus/ency/article/007230.htm
hasDefinitionalManifestation some Neutropenia OR interprets some (hasComponent some 'Segmented neutrophil' OR hasComponent some 'Myelocyte' OR hasComponent some 'Stab form') AND (hasInterpretation some 'Below reference range' OR hasInterpretation some 'Decreased' OR hasInterpretation some 'Abnormal')
Semantic relations used by the query Description interprets Refers to the entity being evaluated or interpreted, when an evaluation, interpretation or “judgment” is intrinsic to the meaning of a concept. hasInterpretation This attribute is grouped with the attribute “Interprets”, and designates the judgment aspect being evaluated or interpreted for a concept (e.g., presence, absence, degree, normality, abnormality, etc.). hasComponent Refers to what is being observed or measured by a procedure. hasDefinitionalManifestation Links disorders to the manifestations (observations) that define them.
(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte
hasDefinitionalManifestation some Neutropenia OR interprets some (hasComponent some 'Segmented neutrophil' OR hasComponent some 'Myelocyte' OR hasComponent some 'Stab form') AND (hasInterpretation some 'Below reference range' OR hasInterpretation some 'Decreased' OR hasInterpretation some 'Abnormal')
Semantic relations used by the query Description interprets Refers to the entity being evaluated or interpreted, when an evaluation, interpretation or “judgment” is intrinsic to the meaning of a concept. hasInterpretation This attribute is grouped with the attribute “Interprets”, and designates the judgment aspect being evaluated or interpreted for a concept (e.g., presence, absence, degree, normality, abnormality, etc.). hasComponent Refers to what is being observed or measured by a procedure. hasDefinitionalManifestation Links disorders to the manifestations (observations) that define them.
(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte
Selecting disorders defined by an abnormally low number of neutrophils
(ex. autoimmune neutropenia, cyclical neutropenia, etc.)
hasDefinitionalManifestation some Neutropenia OR interprets some (hasComponent some 'Segmented neutrophil' OR hasComponent some 'Myelocyte' OR hasComponent some 'Stab form') AND (hasInterpretation some 'Below reference range' OR hasInterpretation some 'Decreased' OR hasInterpretation some 'Abnormal')
Semantic relations used by the query Description interprets Refers to the entity being evaluated or interpreted, when an evaluation, interpretation or “judgment” is intrinsic to the meaning of a concept. hasInterpretation This attribute is grouped with the attribute “Interprets”, and designates the judgment aspect being evaluated or interpreted for a concept (e.g., presence, absence, degree, normality, abnormality, etc.). hasComponent Refers to what is being observed or measured by a procedure. hasDefinitionalManifestation Links disorders to the manifestations (observations) that define them.
Neutrophils or neutrophil granulocytes “are generally referred to as either neutrophils
neutrophils (or segs) and banded neutrophils [also called band neutrophil or stab cell]. They form part of the polymorphonuclear cell family (PMNs) together with basophils and eosinophils.” (*)
(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte
Myelocyte = precursor cell in the development of the granulocyte series. Selecting analyses measuring quantity of neutrophils cells or neutrophils precursor cells
(ex. Myelocyte count, Myelocyte percentage)
hasDefinitionalManifestation some Neutropenia OR interprets some (hasComponent some 'Segmented neutrophil' OR hasComponent some 'Myelocyte' OR hasComponent some 'Stab form') AND (hasInterpretation some 'Below reference range' OR hasInterpretation some 'Decreased' OR hasInterpretation some 'Abnormal')
Semantic relations used by the query Description interprets Refers to the entity being evaluated or interpreted, when an evaluation, interpretation or “judgment” is intrinsic to the meaning of a concept. hasInterpretation This attribute is grouped with the attribute “Interprets”, and designates the judgment aspect being evaluated or interpreted for a concept (e.g., presence, absence, degree, normality, abnormality, etc.). hasComponent Refers to what is being observed or measured by a procedure. hasDefinitionalManifestation Links disorders to the manifestations (observations) that define them.
Neutrophils or neutrophil granulocytes “are generally referred to as either neutrophils
neutrophils (or segs) and banded neutrophils [also called band neutrophil or stab cell]. They form part of the polymorphonuclear cell family (PMNs) together with basophils and eosinophils.” (*)
(*) http://en.wikipedia.org/wiki/Neutrophil_granulocyte
Myelocyte is precursor cell in the development of the granulocyte series. Selecting abnormal / low results of those analyses (ex. Neutrophil count abnormal, Myelocyte
percentage decreased, etc.)
Type (for SMQ) Label Id Meddra Q Label Id Meddra Agranulocytosis 10001507 Y Agranulocytosis 10001507 Autoimmune neutropenia 10055128 Y Autoimmune neutropenia 10055128 Cyclic neutropenia 10053176 Y Band neutrophil count decreased 10057950 Febrile neutropenia 10016288 Y Band neutrophil percentage decreased 10059130 Felty's syndrome 10016386 Y Cyclic neutropenia 10053176 Granulocytopenia 10018687 Y Febrile neutropenia 10016288 Granulocytopenia neonatal 10018688 Y Felty's syndrome 10016386 Idiopathic neutropenia 10051645 Y Granulocytopenia 10018687 Infantile genetic agranulocytosis 10052210 ‐ Granulocytopenia neonatal 10018688 Neutropenia 10029354 Y Idiopathic neutropenia 10051645 Neutropenia neonatal 10029358 Y Myelocyte count decreased 10050986 Neutropenic colitis 10062959 Y Myelocyte percentage decreased 10052227 Neutropenic infection 10059482 Y Neutropenia 10029354 Neutropenic sepsis 10049151 Y Neutropenia neonatal 10029358 13 Neutropenic colitis 10062959 Narrow Agranulocytosis 10001507 Y Neutropenic infection 10059482 Narrow Band neutrophil count decreased 10057950 Y Neutropenic sepsis 10049151 Narrow Band neutrophil percentage decreased 10059130 Y Neutrophil count abnormal 10061313 Narrow Cyclic neutropenia 10053176 Y Neutrophil count decreased 10029366 Narrow Febrile neutropenia 10016288 Y Neutrophil percentage abnormal 10058134 Narrow Idiopathic neutropenia 10051645 Y Neutrophil percentage decreased 10052223 Narrow Neutropenia 10029354 Y Neutrophil percentage decreased 10052223 Narrow Neutropenic infection 10059482 Y Narrow Neutropenic sepsis 10049151 Y Narrow Neutrophil count decreased 10029366 Y Broad Myelocyte percentage decreased 10052227 Y Broad Neutropenia neonatal 10029358 Y Broad Neutrophil count abnormal 10061313 Y Broad Neutrophil percentage decreased 10052223 Y 14
HLT SMQ
Recall 100,0% Precision 59,1% F‐measure 74,3% 100,0% 63,6% 77,8%
HLT SMQ OntoADR OWL Query
Type (for SMQ) Label Id Meddra Q Label Id Meddra Agranulocytosis 10001507 Y Agranulocytosis 10001507 Autoimmune neutropenia 10055128 Y Autoimmune neutropenia 10055128 Cyclic neutropenia 10053176 Y Band neutrophil count decreased 10057950 Febrile neutropenia 10016288 Y Band neutrophil percentage decreased 10059130 Felty's syndrome 10016386 Y Cyclic neutropenia 10053176 Granulocytopenia 10018687 Y Febrile neutropenia 10016288 Granulocytopenia neonatal 10018688 Y Felty's syndrome 10016386 Idiopathic neutropenia 10051645 Y Granulocytopenia 10018687 Infantile genetic agranulocytosis 10052210 ‐ Granulocytopenia neonatal 10018688 Neutropenia 10029354 Y Idiopathic neutropenia 10051645 Neutropenia neonatal 10029358 Y Myelocyte count decreased 10050986 Neutropenic colitis 10062959 Y Myelocyte percentage decreased 10052227 Neutropenic infection 10059482 Y Neutropenia 10029354 Neutropenic sepsis 10049151 Y Neutropenia neonatal 10029358 13 Neutropenic colitis 10062959 Narrow Agranulocytosis 10001507 Y Neutropenic infection 10059482 Narrow Band neutrophil count decreased 10057950 Y Neutropenic sepsis 10049151 Narrow Band neutrophil percentage decreased 10059130 Y Neutrophil count abnormal 10061313 Narrow Cyclic neutropenia 10053176 Y Neutrophil count decreased 10029366 Narrow Febrile neutropenia 10016288 Y Neutrophil percentage abnormal 10058134 Narrow Idiopathic neutropenia 10051645 Y Neutrophil percentage decreased 10052223 Narrow Neutropenia 10029354 Y Neutrophil percentage decreased 10052223 Narrow Neutropenic infection 10059482 Y Narrow Neutropenic sepsis 10049151 Y Narrow Neutrophil count decreased 10029366 Y Broad Myelocyte percentage decreased 10052227 Y Broad Neutropenia neonatal 10029358 Y Broad Neutrophil count abnormal 10061313 Y Broad Neutrophil percentage decreased 10052223 Y 14
HLT SMQ
Recall 100,0% Precision 59,1% F‐measure 74,3% 100,0% 63,6% 77,8%
HLT SMQ OntoADR OWL Query
All PTs present in the handmade reference groupings can be selected automatically
Label Id Meddra Q Label Id Meddra Acquired epidermolysis bullosa 10056508 Y Acne pustular 10000513 Application site vesicles 10048941 Y Acquired epidermolysis bullosa 10056508 Benign familial pemphigus 10004265 ‐ Acute generalised exanthematous pustulosis 10048799 Blister 10005191 Y Application site pustules 10049044 Blood blister 10005372 Y Application site vesicles 10048941 Bullous impetigo 10006563 Y Blister 10005191 Cervical bulla 10050019 Y Blister infected 10005192 Coma blister 10069827 Y Blood blister 10005372 Dermatitis bullous 10012441 Y Bovine pustular stomatitis virus infection 10006048 Dermatitis herpetiformis 10012468 Y Bullous impetigo 10006563 Diabetic bullosis 10062356 Y Cervical bulla 10050019 Epidermolysis 10053177 N Coma blister 10069827 Epidermolysis bullosa 10014989 Y Dermatitis bullous 10012441 Erythema multiforme 10015218 N Dermatitis herpetiformis 10012468 Herpes gestationis 10019939 N Diabetic bullosis 10062356 Implant site vesicles 10063870 Y Eczema herpeticum 10014197 Incision site blister 10067990 Y Eosinophilic pustular folliculitis 10052834 Infusion site vesicles 10065491 Y Eosinophilic pustulosis 10049172 Injection site vesicles 10022111 Y Epidermolysis bullosa 10014989 Linear IgA disease 10024515 N Heat rash 10019343 Lip blister 10049307 Y Herpes zoster oticus 10063491 Ocular pemphigoid 10067776 Y Impetigo herpetiformis 10021534 Oculomucocutaneous syndrome 10030081 Y Implant site pustules 10063866 Paraneoplastic pemphigus 10057056 Y Implant site vesicles 10063870 Pemphigoid 10034277 Y Incision site blister 10067990 Pemphigus 10034280 Y Infusion site pustule 10065487 Penile blister 10052898 Y Infusion site vesicles 10065491 Porphyria non‐acute 10036186 N Injection site pustule 10054994 Pseudoporphyria 10037145 N Injection site vesicles 10022111 Staphylococcal scalded skin syndrome 10041929 N Lip blister 10049307 Stevens‐Johnson syndrome 10042033 N Ocular pemphigoid 10067776 Toxic epidermal necrolysis 10044223 N Oculomucocutaneous syndrome 10030081 Vaccination site vesicles 10069623 Y Paraneoplastic pemphigus 10057056 TOTAL 23 Pemphigoid 10034277 Pemphigus 10034280 Penile blister 10052898 Pustular psoriasis 10037575 Rash follicular 10037857 Rash pustular 10037888 SAPHO syndrome 10051316 Subcorneal pustular dermatosis 10042342 Urticaria vesiculosa 10046755 Vaccination site pustule 10066047 Vaccination site vesicles 10069623
Recall 71,9% Precision 52,3% F‐measure 60,5%
HLT Bullous conditions OntoADR OWL Query
For other safety topics some PTs can’t be caught with the query method
19
Recall, precision and F‐measure rates of automatic OntoADR groupings. SELECT (third column) indicates that a manual selection of PTs has been made in the MedDRA grouping taken as gold standard. (++) MedDRA reference groupings having a perfect semantic match with the safety topic. (+) MedDRA reference groupings having an imperfect semantic match with the safety topic (broader or narrower topic).
20
Recall, precision and F‐measure rates of automatic OntoADR groupings. SELECT (third column) indicates that a manual selection of PTs has been made in the MedDRA grouping taken as gold standard. (++) MedDRA reference groupings having a perfect semantic match with the safety topic. (+) MedDRA reference groupings having an imperfect semantic match with the safety topic (broader or narrower topic).
21
Recall, precision and F‐measure rates of automatic OntoADR groupings. SELECT (third column) indicates that a manual selection of PTs has been made in the MedDRA grouping taken as gold standard. (++) MedDRA reference groupings having a perfect semantic match with the safety topic. (+) MedDRA reference groupings having an imperfect semantic match with the safety topic (broader or narrower topic).
22
Recall, precision and F‐measure rates of automatic OntoADR groupings. SELECT (third column) indicates that a manual selection of PTs has been made in the MedDRA grouping taken as gold standard. (++) MedDRA reference groupings having a perfect semantic match with the safety topic. (+) MedDRA reference groupings having an imperfect semantic match with the safety topic (broader or narrower topic).
Results demonstrate that OntoADR and this OWL query-based grouping method can efficiently support the realization of automatic MedDRA terms groupings. Promising result:
manually
Next step: (a) Some terms of the handmade reference MedDRA groupings not retrieved by our automatic queries, and (b) new terms are caught. Does this difference in content has an impact (and if so is it positive?) on the signals detected by traditional statistical methods? Further study to address the question of the reliability of those query-based groupings for signal detection (comparison with standard MedDRA groupings - first of all of SMQs - signal detection performances).
Forthcoming: Souvignet, J., Declerck, G., Trombert, B., Rodrigues, J.M., Jaulent, M.C., Bousquet, C. Evaluation of automated term groupings for detecting anaphylactic shock signals with drugs. AMIA 2012, American Medical Informatics Association Annual Symposium, 3‐7 novembre 2012, Chicago (US).
23
24
25
Only about 50% of MedDRA terms are mapped with Snomed-CT concepts within UMLS. A complete mapping of MedDRA seems very difficult to realize. Especially most MedDRA terms have no direct equivalent in Snomed-CT: their meaning must be decomposed to be mapped. Ex.: MedDRA « Joint dislocation postoperative » |10049912| : No direct Snomed-CT equivalent Decompositon of MedDRA term to perform the mapping:
A single MedDRA term can be mapped to several Snomed-CT concepts within the same UMLS concept. UMLS embed one-to-one mappings but also one-to-several mappings. Necessity to make manual selection to get a unique definition.
26
UMLS synonymy links between MedDRA terms and Snomed-CT concepts are not always correct.
‘Neoplasm of uncertain behavior of pericardium’ |94997003| and ‘Neoplasm of pericardium’ |126734005|
spondylitis’ |399096009|, ‘Inflammatory spondylopathy’ |202649003| and ‘Spondylitis’ |84172003|
lymphatic systems. Snomed-CT ‘Vascular disorder’ |27550009| only refers to disorders of blood vessels (‘Disorder of lymphatic system’ is used to refer to lymphatic system troubles) With our mapping method, 39.8 % of MedDRA 13.0 terms (i.e. 8305 terms) were mapped via a one-to-one mapping and 9.5 % (i.e. 1984) were mapped via a one-to-several mapping.
27