Grouping pharmacovigilance terms with semantic distance M Dupuch 1 , - - PowerPoint PPT Presentation

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Grouping pharmacovigilance terms with semantic distance

M Dupuch1,2, M Lerch3, A Jamet2,4, MC Jaulent1,2, R Fescharek5 and N Grabar6

(1) Universit´ e Pierre et Marie Curie - Paris6, Paris, F-75006 France; (2) INSERM, U872 eq. 20, Paris, F-75006 France; (3) Consulting & Coaching, Berlin, Germany; (4) HEGP, AP-HP, Paris, France; (5) CSL Behring GmbH, Marburg, Germany; (6) CNRS UMR 8163 STL, Universit´ e Lille 3, France Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Plan of the presentation

Context Objective Material Methods Results and Discussion Conclusion and Perspectives

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Pharmacovigilance

Pharmacovigilance: activity related to the collection, analysis and prevention of adverse drug reactions (ADRs) ADRs are coded with 2 terminologies:

⇒ MedDRA (Medical Dictionnary for Drug Regulatory Activities) WHO-ART (World Health Organization - Adverse Reaction Terminology)

Signal: unexpected or not well documented relation between drugs and ADRs

Exploitation of statistical methods for detection of signals (Bate & al, 1998;Meyboom & al, 2002) Exploitation of SMQs (Standardized MedDRA Queries)

Groupings of terms associated to a given safety topic

⇒ Grouping of related cases of pharmacovigilance

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Exploitation and evaluation of SMQs

SMQs are created manually by experts

Structure of MedDRA Scientific litterature ⇒ Long and tedious process

Exploitation and evaluation of SMQs:

Consequent filtering and evaluation of cases is required:

High sensibility and over-inclusiveness (Mozzicato, 2007; Pearson & al, 2009)

Silences:

Important terms may not be included (Pearson & al, 2009)

⇒ Propose automatic methods for the creation of SMQs or of new groupings

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Objective

Hypothesis: exploitation of the semantic distance

Depends on the number of edges (the shortest path)

Abdominal Disorder of abscsess abdomen Disorder of trunk

Subclass of subclass of

sp = 2 sp = 1

Previous exploitation of semantic distance with pharmacovigilance terms:

subsets of MedDRA terms (Bousquet & al, 2005) subsets of WHO-ART terms (Iavindrasana & al, 2006) ⇒ No evaluation of the groupings with the SMQs

Objective: continue to adapt the semantic distance to PV terms

Creation of groupings of terms and of new SMQs Exploitation of the whole set of MedDRA terms Comparison with the SMQs

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Material

MedDRA

Level Expanded form Nb Terms SOC System Organ Class 26 HLGT High Level Group Terms 332 HLT High Level Terms 1,688 PT Prefered Terms 18,209 LLT Lowest Level Terms 66,587 Two kinds of material derived from MedDRA:

• ntoEIM resource

SMQs

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Material

• ntoEIM

OntoEIM resource (Alecu & al, 2008): Projection of MedDRA terms on SNOMED CT

Currently 46% of MedDRA terms are aligned

⇒ Finer-grained hierarchy of MedDRA terms ⇒ Formal definitions (FD) as extracted from SNOMED CT:

Morphology M, topography T, causality C and expression E

Arsenical keratosis (diagnosis ADR)

⇒ M: Squamous cell neoplasm; Morphologically abnormal structure ⇒ T: Skin structure; Structure of skin and or surface apithelium C: Arsenic AND OR arsenic compound E: Abnormal keratinization

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Material

SMQs

SMQs (Standartized MedDRA Queries) Groupings of terms related to a safety topic

Acute renal failure, Hepatic disorders, Thrombocytopenia...

Helpful for the searching of close pharmacovigilance cases Created manually by experts 84 SMQs exist currently SMQs include PT and LLT terms Gold standard: 9 SMQs (ADRs leading to hospitalization, vital prognosis and death)

Acute renal failure, Agranulocytosis, Anaphylactic reaction, Cytopenia, Gastrointestinal haemorrhages, Peripheral neuropathy, Rhabdomyolysis, Severe cutaneous adverse reaction, Thrombocytopenia

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Methods

Tree-step method:

1 Optimisation of the alignment and of formal definitions 2 Computing of the semantic distance and grouping of terms 3 Evaluation of the generated groupings of terms Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

• 1. Optimisation of the alignment and of formal definitions

46% of MedDRA terms are aligned with SNOMED CT

51.6% of PTs and 35.4% of LLTs ⇒ Optimize the alignment and improve the coverage

Abdominal abscess

T: Abdominal cavity structure M: Abscess morphology T: Abdominal cavity structure M: Abscess morphology T: Abdominal cavity structure

phlegmon Abdominal abscess Intra abdominal

M: Abscess morphology

LLT PT

PTs and LLTs aligned distOntoEIM

FD PT FD LLT

Abdominal abscess

T: Abdominal cavity structure M: Abscess morphology T: Abdominal cavity structure M: Abscess morphology T: Abdominal cavity structure M: Abscess morphology

phlegmon Abdominal abscess Intra abdominal

+1 +1

PT LLT

+1 +1

PTs aligned; LLTs not aligned Transfer FD PTs ⇒ LLTs sp(LLT) = sp(PT) + 1

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

• 1. Optimisation of the alignment and of formal definitions

Abdominal abscess

T: Abdominal cavity structure M: Abscess morphology

phlegmon Abdominal

T: Abdominal cavity structure M: Abscess morphology

LLT PT

−1 −1

LLTs aligned; PTs not aligned Transfer FD LLTs ⇒ PTs sp(PT) = sp(LLT) − 1

Abdominal abscess phlegmon Abdominal abscess Intra abdominal LLT PT

LLTs and PTs not aligned ⇒ Semantic distance not computed Improvement of the alignment of MedDRA terms:

+10% for PTs (61.6%) +30% for LLTs (65.4%)

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

• 2. Computing of semantic distance and grouping of terms

Computing of the semantic distance (Rada et al, 1989) The shortest path between two concepts distRada = sp(c1, c2) The shortest path is the sum of all its edges Each edge is equal to 1 Parameters tested: One axis (ADRs) Three axes (ADR + FD) All SMQ terms Aligned SMQ terms Best grouping Merging of n best groupings

abscess Abdominal abscess Pharyngal

1 1 1 1 1 1 1 1 1 1 1 1 1 1 pcc_D (Abdominal abscess, Pharyngeal abscess) = 4 pcc_M (Abdominal abscsess, Pharyngeal abscsess) = 0 pcc_T (Abdominal abscsess, Pharyngeal abscsess) = 10

T: Abdominal cavity structure M: Abscess morphology pcc_M = 0 M: Abscess morphology T: Neck structure pcc_D = 4 pcc_T = 10

spADR = 4, spT = 10 et spM = 0

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

• 2. Computing of semantic distance and grouping of terms

Each axis is weighted (Petiot & al, 1996)

distontoEIM(A, B) =

• i∈{ADR,M,T}

Wi ∗ sp(Ai, Bi)

• j∈{ADR,M,T}

Wj

WADR = 1, WM = 2 et WT = 1

⇒ M is the most important

Generation of semi-matrix

distontoEIM between all PT and LLT terms

distontoEIM(Abdominal abscess, Pharyngeal abscess) = 3.5 Grouping: distance <= 2

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

• 3. Evaluation of the generated groupings of terms

Three evaluation measures:

precision P: number of relevant grouped terms as a percentage of the total number of the grouped terms recall R: number of relevant grouped terms as a percentage of the number of terms in the corresponding SMQ f-measure F: the harmonic mean of P and R

Association between the generated groupings and the SMQs

⇒ precision

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Results and Discussion

20 40 60 80 100 1a−bc 1a−ba 3a−bc 3a−ba 1a−mc 1a−ma 3a−mc 3a−ma max Precision min 20 40 60 80 100 1a−bc 1a−ba 3a−bc 3a−ba 1a−mc 1a−ma 3a−mc 3a−ma max Recall min 20 40 60 80 100 1a−bc 1a−ba 3a−bc 3a−ba 1a−mc 1a−ma 3a−mc 3a−ma max F−measure min

precision recall f-measure Merging of groupings: increasing of the overall performance 1 axis vs 3 axes: best performances with 1 axis

incompleteness of the formal definitions

Set of aligned terms: R and F increase but P decreases Min-max intervals very large: variability between the SMQs

various strategies needed according to the safety topics

Best results: 1 axis, merged groupings

high precision (expectation of the experts)

Marie Dupuch

Context Objective Material Method Results and Discussion Conclusion and Perspectives

Conclusion and Perspectives

Method for the creation of groupings of ADR terms Optimization of alignment: +10% for PTs, +30% for LLTs Different parameters of the method:

⇒ Best results: 1 axis, merged groupings

Evaluation: high precision (expectation of the experts) Perspectives:

Improvement of the alignment of the MedDRA terms Broad and narrow versions of SMQs Adjustment of variables (edge weights, coefficient of axes...) Other measures for semantic distance (Leacock, Zhong...) Other clustering methods (hierarchical, partitionning) NLP methods for enriching and refining the groupings ⇒ Definition of different strategies for different safety topics

Marie Dupuch