Marie-Christine Jaulent Iulian Alecu SPIM- UMRS872,EQ20 - France MIE 2009 – Sarajevo - 31 aout 2009
Agenda History of the project Pharmacovigilance and Signal detection The ontological resource: OntoEIM The web service PharmARTS Evaluation of the ontological resource Context Design of the study Methodology of evaluation Current results Evolution and maintenance of the ontology
Pharmacovigilance and Signal Detection Pharmacovigilance Set of procedures for the identification, evaluation and prevention of ADR (adverse drug reaction) risk ADR are coded in databases • terminologies WHO-ART (WHO) & MedDRA (FDA – MSSO) Signal detection Signal ↔ possible cause {Drug → ADR} (OMS) Data mining - automatic analysis need prior automatic grouping Limits of terminology’ structure for grouping [Bousquet et a. 2005] Lack of polyhierarchy MSSO : manual answer to the problem, SMQ • Example: « acute pancreatitis » : 47 prefered terms, 320 terms including synonyms (Cullen's sign, Hereditary pancreatitis,Ischaemic pancreatitis, Blood amylase increased, Pancreatic enzymes increased, Urine amylase abnormal, Blood bilirubin increased, Gastrointestinal pain, Nausea, …) • 75 SMQs in march 2009
An ontology to group cases « on the fly » OntoEIM [Alecu et. Al., 2008]* 90000 concepts (45000 SNOMED_CT concepts = 10% of all SNOMED_CT) WHO-ART MedDRA Snomed_CT * Alecu I, Bousquet C, Jaulent MC. A case report: using SNOMED CT for grouping Adverse Drug Reactions Terms. BMC Med Inform Decis Mak . 2008 Oct 27;8 Suppl 1:S4.
PharmARTS [Alecu et. Al., Medinfo 2007], US patent in 2007 Browse the resource for coding purpose, Build queries to group terms Retrieve cases in a pharmacovigilance database
Evaluation of OntoEIM Context: Define « on the fly » a medical condition by querying the ontology and grouping concepts Question: Is the ontology appropriate to propose relevant set of concepts for a given medical condition? Evaluation of ontologies [Portzel 2004; Rogers 2006; Cornet 2008] internal and external the relevance of the vocabulary describing the concepts the relevance of the “is-a” hierarchy the relevance of the semantic relations OntoEIM is evaluated according to its purpose Rogers 2006: One significant problem for ontology quality assurance is the lack of a gold standard against which to determine the correctness of the ontology and its suitability for the given purpose Definition of a precise gold standard: SMQ
Design of the study Medical Built OntoEIM condition query return SMQ COMPARE? Gold Standard
Evaluation of the query Modelisation for each medical condition Group of individuals: Set of possible terms (MedDRA) Definition of the condition: term present in SMQ (SMQ +) Definition of the test (a query). We say that the test is: • Positive if the term is returned by the query (Q+) • Negative if the term is not returned by the query (Q-) SMQ + SMQ - c Q+ a b a b Q- c d Sensibility: a / (a+c) Specificity: d / (d+b) ≈ 1
Construction of the query A two steps automatic process Selection of « candidate concepts » in OntoEIM Construction of an optimal query
Selection of « candidate concepts » in OntoEIM Correspondance Example C • SMQ = ACUTE RENAL FAILURE • C = ACUTE RENAL FAILURE Enlargment C’ Example : 1) SMQ = CONVULSIONS C 2) C = CONVULSIONS 3) C’ = Seizures_incl_subtypes Composition C2 C1 Example : 1) SMQ = Asthma/bronchospasm 2) C1 = Asthma 3) C2 = bronchospasm
Construction of an optimal query For each candidate concept, we calculate a, b and c If there are several candidate concepts, we build as queries as possible using the union operator C1, C2 Q1=C1; Q2=C2; Q3=C1+C2 c a b Final query Q = Q such that max (a) and min (b+c) < λ λ is an heuristic
Results Results for 24 SMQs Number of terms in the SMQ Sensitivity Mean Range Mean Range 23,5 [7 ; 47] 0,82 [0,45 ; 1] List of 24 SMQs Angioedema ; Acute central respiratory depression ; Acute pancreatitis; Acute renal failure ; Agranulocytosis ; Anaphylactic reaction; Asthma/bronchospasm ; Cardiac failure; Haemorrhagic cerebrovascular condition; Convulsions; Guillain-Barre syndrome ; Dyslipidaemia ; Hostility/aggression ; Interstitial lung disease; Lack of efficacy/effect; Neuroleptic malignant syndrome; Pseudomembranous colitis ; Peripheral neuropathy ; Pulmonary hypertension ; Retroperitoneal fibrosis ; Rhabdomyolysis/ myopathy ; Severe cutaneous adverse reactions; Torsade de pointes/QT prolongation ; hyperglycemie/ diabete mellitus
Discussion Typology of errors b ≠ 0 . The term is missing in the SMQ. SMQ « angioedema », term W:OEDEMA_CIRCUMORAL. b ≠ 0. There is a taxonomic error in the ontology . SMQ “acute pancreatitis”, term W :PANCREATITIS_CHRONIC, c ≠ 0. Formal concept definitions are not complete . c ≠ 0. Semantic relations are missing in the ontology the relation “evoke” between an exam result and a diagnosis does not exist in the ontology
Conclusion Evaluation of an existing resource Specific purpose : prior grouping of terms to improve the performances of signal detection algorithms in pharmacivigilance Evaluation by comparison with the existing SMQs Implementation of the approach Reusability Ontology evolution : The method does not support the development of the ontology but allows identifying how it has to evolve by finding the corrections that have to be made (missing definitions or missing relations, wrong definitions or wrong relations) Ontology maintenance: measure if the ontology remains adequate when there is a new context (construction of new SMQs by the MSSO)
Perspectives Continue the evaluation for the other SMQs Collect manually the classification and corrections of errors done during the evaluation in order to extract, if possible, some rules that could help the correction process. Design tools and interfaces to assist the resolution of the errors revealed by the method Define medical conditions « on the fly » (design of new SMQs)
Recommend
More recommend
