Processing the Scope of Negation and Modality Cues in Biomedical - - PowerPoint PPT Presentation

Processing the Scope of Negation and Modality Cues in Biomedical Texts

Roser Morante, Walter Daelemans CNTS-Language Technology Group University of Antwerp

1

Framework

• The BIOGRAPH project (www.biograph.be)

University of Antwerp:

• Text Mining: CNTS, Department of Linguistics,

Walter Daelemans

• Data Mining: ADReM, Department of

Mathematics and Computer Science Bart Goethals

• Genetics: AMG, Department of Molecular

Genetics, Jurgen Del-Favero

2

Framework

• The BIOGRAPH project aims at:
• Assisting researchers in ranking candidate disease

causing genes by putting forward a new methodology for combined text analysis and data mining from heterogeneous information sources

• Mining biomedical texts: providing accurate relations

automatically extracted from text and weighted according to their reliability

• Treatment of negation, modality and quantification

3

Framework

The BIOGRAPH flow

4

• Candidate region
• Gene responsible for a disease (e.g. schizophrenia or

Alzheimer) is in a known area of the genome

• Many genes (> 200) are in this candidate region
• Experimental validation is needed
• Very expensive in time and cost
• Combine information in literature and in databases!
• Which genes in the candidate region could be most

relevant for the disease and why?

• Provide a prioritization (ranking problem)

Gene Prioritization

5

Event Extraction

MEDLINE:7747440 Epstein-Barr virus replicative gene transcription during de novo infection of human thymocytes: simultaneous early expression of BZLF-1 and its repressor RAZ. Epstein-Barr virus (EBV) is known to infect B cells and epithelial cells. We and others have shown that EBV can also infect a subset of thymocytes. Infection of thymocytes was accompanied by the appearance of linear EBV genome within 8 hr of infection. Circularization of the EBV genome was not detected. This is in contrast to the infection in B cells where the genome can circularize within 24 hr of infection. The appearance of the BamHI ZLF-1 gene product, ZEBRA, by RT-PCR, was observed within 8 hr of infection. The appearance of a novel fusion transcript (RAZ), which comprised regions of the BZLF-1 locus and the adjacent BRLF-1 locus, was detected by RT-PCR. ZEBRA protein was also identified in infected thymocytes by immunoprecipitation. In addition, we demonstrated that the EBNA-1 gene in infected thymocytes was transcribed from the Fp promoter, rather than from the Cp/Wp promoter which is used in latently infected B cells. Transcripts encoding gp350/220, the major coat protein of EBV, were identified, but we did not find any evidence of transcription from the LMP-2A or EBER-1 loci in infected

• thymocytes. These observations suggest that de novo EBV infection of thymocytes differs

from infection of B cells. The main difference is that with thymocytes, no evidence could be found that the virus ever circularizes. Rather, EBV remains in a linear configuration from which replicative genes are transcribed.

6

Event Extraction

MEDLINE:7747440 ... In addition, we demonstrated that the EBNA-1 gene in infected thymocytes was transcribed from the Fp promoter, rather than from the Cp/Wp promoter which is used in latently infected B cells.

Transcripts encoding gp350/220, the major coat protein of EBV, were identified, but we did not find any evidence of transcription from the LMP-2A or EBER-1 loci in infected

• thymocytes. These observations suggest that de novo EBV infection of thymocytes differs from

infection of B cells.

<event id="E10" source="7747440" neg="1" spec="1"> <predicate type="Transcription" begin="1216" end="1229"> transcription </predicate> <patient type="Theme" begin="1239" end="1245"> LMP-2A </patient> </event>

7

Contents

• Motivation
• Negation
• Task description
• Related work
• Corpus
• System description
• Results
• Modality
• Related work
• Results
• Negation vs. modality
• Conclusions
• Further Research

8

Motivation

• Extracted information that falls in the scope of

hedge or negation cues cannot be presented as factual information

• Vincze et al. (2008) report that 17.70% of

the sentences in the BioScope corpus contain hedge cues and 13 % negation cues

• Light et al. (2004) estimate that 11% of

sentences in MEDLINE abstracts contain speculative fragments

9

Finding the scope of negation

• Finding the scope of a negation cue means

determining at a sentence level which words in the sentence are affected by the negation(s)

Analysis at the phenotype and genetic level showed that

lack of CD5 expression was due neither to segregation of human autosome 11, on which

the CD5 gene has been mapped, nor to deletion of

the CD5 structural gene.

10

Related work

• Most of the related work focuses on detecting

whether a term is negated or not

• Rule or regular expression based systems like NegEx

(Chapman et al. 2001) and NegFinder (Mutalik et al. 2001)

• Machine learning systems like Averbuch et al. (2004)
• Huang and Lowe (2007) develop a hybrid system that

combines regular expression matching with parsing in

• rder to locate negated concepts

11

Corpus

12

PMA treatment, and <xcope id=“X1.4.1”> <cue type=“negation'' ref="X1.4.1"> not </cue> retinoic acid treatment of the U937 cells</xcope> acts in inducing NF-KB expression in the nuclei.

Corpus

• Medical and biological texts annotated with

information about negation and speculation

</xcope> <xcope id=“X1.4.1”> </cue> <cue type=“negation'' ref="X1.4.1">

282243 60935 41985 #Words 11871 2670 6383 #Sent. 1273 9 1954 #Docs. Abstracts Papers Clinical

• Corpora

13

Experimental Setting

• Abstracts corpus:
• 10 fold cross-validation experiments
• Clinical and papers corpora:

robustness test

• Training on abstracts
• Testing on clinical and papers

14

System Description

• We model the scope finding task as two

consecutive classification tasks:

• Finding negation cues: a token is classified as being at

the beginning of a negation signal, inside or outside

• Finding the scope: a token is classified as being the

first element of a scope sequence, the last, or neither

• Supervised machine learning approach

15

System Architecture

16

Preprocessing

17

Finding Negation Cues

• We filter out negation cues that are

unambiguous in the training corpus (17 out of 30)

• For the rest, a classifier predicts

whether a token is the first token of a negation signal, inside or outside of it

• Algorithm : IGTREE as implemented

in TiMBL (Daelemans et al. 2007)

• Instances represent all tokens in a

sentence

• Features about the token in focus and

its context

18

Features negation cue finding

• Of the token
• Lemma, word, POS and IOB chunk tag
• Of the token context
• Word, POS and IOB chunk tag of 3 tokens

to the right and 3 to the left

19

Ambiguous Negation Cues

In Abstracts Corpus

20 97.42 88.03 88.09 F1 90.70 79.42 94.46 IAA 97.53 92.46 95.17 RECALL 97.31 84.01 82.00 PREC Clinical Papers Abstracts BASELINE

Results

• Baseline: tagging as negation signals tokens that are

negation signals at least in 50% of the occurrences in the training corpus

BASELINE TOKENS absence, absent, cannot, could not, fail, failure, impossible, instead of, lack, miss, neither, never, no, none, nor, not, rather than, unable, with the exception of, without

97.71 (+0.29) 91.25 (+3.22) 91.20 (+3.11) F1 98.09 95.72 98.75 RECALL 97.33 87.18 84.72 PREC Clinical Papers Abstracts SYSTEM

21

Results system vs. baseline in abstracts corpus

• The system performs better

22

Results in the three corpora

• The system is portable

23

Discussion

• Cause of lower recall on papers corpus:

93.68 53.22 Papers 91.22 6.72 Clinical 98.25 58.89 Abstracts % classified correctly % negation signals NOT

• Errors: not is classified as negation signal

However, programs for tRNA identification [...] do not necessarily perform well on unknown ones The evaluation of this ratio is difficult because not all true interactions are known

24

Finding Scopes

• Three classifiers predict whether a

token is the first token in the scope sequence, the last or neither

• MBL (Daelemans et al. 2007)
• SVMlight (Joachims 1999)
• CRF++ (Lafferty et al. 2001)
• A fourth classifier predicts the same

taking as input the output of the previous classifiers

• CRF++
• The features used by the object

classifiers and the metalearner are different

25

Finding Scopes

26

Finding Scopes

• Previous attempts: lower results
• Chunk-based classification, instead of word-based
• BIO classification of tokens (EMNLP’08) instead of

FOL (First, Other, Last)

• Single classifier approach, instead of metalearner

27

Features Scope Finding Object Classifiers

• Of the negation signal: Chain of words.
• Of the paired token: Lemma, POS, chunk IOB tag, type of chunk;

lemma of the second and third tokens to the left; lemma, POS, chunk IOB tag, and type of chunk of the first token to the left and three tokens to the right; first word, last word, chain of words, and chain of POSs of the chunk of the paired token and of two chunks to the left and two chunks to the right.

• Of the tokens between the negation signal and the token in

focus: Chain of POS types, distance in number of tokens, and chain

• f chunk IOB tags.
• Others: A feature indicating the location of the token relative to the

negation signal (pre, post, same).

28

Features Scope Finding Metalearner

• Of the negation signal: Chain of words, chain of POS, word of the two

tokens to the right and two tokens to the left, token number divided by the total number of tokens in the sentence.

• Of the paired token: Lemma, POS, word of two tokens to the right and

two tokens to the left, token number divided by the total number of tokens in the sentence.

• Of the tokens between the negation signal and the token in focus:

Binary features indicating if there are commas, colons, semicolons, verbal phrases or one of the following words between the negation signal and the token in focus: Whereas, but, although, nevertheless, notwithstanding, however, consequently, hence, therefore, thus, instead, otherwise, alternatively, furthermore, moreover.

• About the predictions of the three classifiers: prediction, previous and

next predictions of each of the classifiers, full sequence of previous and full sequence of next predictions of each of the classifiers.

• Others: A feature indicating the location of the token relative to the

negation signal (pre, post, same).

29

Parameters Classifiers

• TiMBL: IB1
• Similarity metric: overlap
• Feature weighting: gain ratio
• 7 k-nn
• Weighting class vote of neighbors as a function of their inverse

linear distance

• SVM
• Classification
• Cost factor: 1
• Biased hyperplane
• Linear kernel function
• CRF
• Regularisation algorithm L2 for

training

• Cut-off threshold of features: 1
• Unchanged hyper-parameter

30

Post-processing

• Scope is always a consecutive block of scope tokens,

including the negation signal

• The classifiers predict the first and last token of the scope

sequence:

• None or more than one FIRST and one LAST elements are

predicted

• In the post-processing we apply some rules to select one

FIRST and one LAST token

Example:

• If more than one token has been predicted as FIRST, take as FIRST the

first token of the negation signal

31

Results

• Baseline: calculating the average length of the scope to the

right of the negation signal and tagging that number of tokens as scope tokens

• Motivation: 85.70 % of scopes to the right

12.95 4.76 7.11 PCS 62.27 24.86 37.45 PCS-2 76.29 70.86 92.46 IAA Clinical Papers Abstracts BASELINE 71.21 70.75 Clinical 41.00 66.07 PCS 44.44 66.93 PCS-2 Papers Abstracts SYSTEM +16.74 +16.52 Clinical +9.79 +9.26 Papers +7.17 +7.29 Abstracts PCS PCS-2 SYSTEM gold negs

32

Results on the abstracts corpus

The system performs clearly better than baseline There is a higher upperbound calculated with gold standard negation signals

33

The system is portable Lower results in the papers corpus

Results on the three corpora

34

Discussion

• Clinical reports are easier to process than

abstracts and papers

• Negation signal no is very frequent (76.65 %) and

has a high PCS (73.10 %)

No findings to account for symptoms No signs of tuberculosis

• Sentences are shorter than in abstracts and papers
• Average length: 7.8 tokens vs. 26.43 and 26.24
• 75.85 % of the sentences have 10 or less tokens

35

Discussion

• Papers are more difficult to process than

abstracts

• Negation signal not is frequent (53.22%) and has

a low PCS (39.50) in papers. Why?

16.41 23.28 % Scopes left 8.85 6.45

• Av. scope length

8.82 5.60

• Av. scope left

14.29 25.56 Ambiguity (%¬neg) Abstracts Papers

NOT

36

PCS results of the metalearner compared to the object classifiers

The metalearner performs better than the three object classifiers (except SVMs on the clinical corpus)

37

Finding the scope of modality

• Finding the scope of a hedge cue means

determining at a sentence level which words in the sentence are affected by the hedge cues(s) These results [suggest that expression of c-jun, jun B and jun D genes [might be involved in terminal granulocyte differentiation

[or in regulating granulocyte functionality]]].

38

Related Work

• Theoretical descriptions that define hedging and

modality (Lakoff 1972, Palmer 1986) based on corpora (Hyland 1998, Saurí et al. 2006, Thompson et al. 2008)

• Machine learning experiments that focus on

classifying a sentence into speculative or definite (Medlock and Briscoe 2007, Medlock 2008, Szarvas 2008, Kilicoglu and Bergler 2008)

39

Related work

• The system that we present here is based on the

system developed for processing the scope of negation cues

• Our goal is to check whether the same approach

can be applied to processing hedge cues

40

System Architecture

S A M E S Y S T E M

41

Preprocessing

42

Finding Hedge Cues

• A classifier predicts whether a token

is at the beginning of a hedge cue, inside or outside of it

• Algorithm : IGTREE as

implemented in TiMBL (Daelemans et al. 2007)

• Instances represent all tokens in a

sentence

• Features about the token in focus

and its context

S A M E S Y S T E M

43

Ambiguity in Hedge Cues

Sample from Abstracts Corpus

# Hedge cues: 110 # Non ambiguous hedge cues: 40

44 50.57 57.60 62.67 F1 84.01 77.60 79.12 IAA 40.78 61.21 71.77 RECALL 66.55 54.39 55.62 PREC Clinical Papers Abstracts BASELINE

Results

41.92 71.59 84.77 F1 27.51 68.18 79.84 RECALL 88.10 75.35 90.81 PREC Clinical Papers Abstracts SYSTEM

• Baseline: tagging as hedge cues a list of words extracted

from the abstracts corpus

BASELINE TOKENS appear, apparent, apparently, believe, estimate, hypothesis, hypothesize, if, imply, likely, may, might, or, perhaps, possible, possibly, postulate, potentially, presumably, probably, propose, putatitve, should, seem, speculate, suggest, support, suppose, suspect, think, uncertain, unclear, unknown, unlikely, whether, would

45

Results system vs. baseline in abstracts corpus

• The system performs better than baseline, with a main increase

in precision (+35.19)

46

Results in the three corpora

• The system is portable in terms of precision, but less so in terms
• f recall, which decreases (-13.27) in the clinical corpus. Why?

47

Discussion

• Cause of lower recall on clinical corpus:

276 27 118 # as hedge 24.62 4.04 4.42 % of hedges 98.22 16.99 11.29 % as hedge 281 153 1062 total # 0.007 0.137 0.129 recall Papers Clinical Abstracts OR

• The use of OR as hedge cue is difficult to interpret

+CUE: Nucleotide sequence and PCR analyses demonstrated the presence

• f novel duplications or deletions involving the NF-kappa B motif.
• CUE: In nuclear extracts from monocytes or macrophages, induction of

NF-KB occurred only if the cells were previously infected with HIV-1. (= AND)

48

Finding Scopes

• Three classifiers predict whether a token

is the first token in the scope sequence, the last or neither

• MBL (Daelemans et al. 2007)
• SVMlight (Joachims 1999)
• CRF++ (Lafferty et al. 2001)
• A fourth classifier predicts the same

taking as input the output of the previous classifiers

• CRF++
• The features used by the object

classifiers and the metalearner are different

S A M E S Y S T E M

49

Finding Scopes

50

Postprocessing

• Scope is always a consecutive block of scope tokens,

including the negation signal

• The classifiers predict the first and last token of the scope

sequence:

• None or more than one FIRST and one LAST elements

might be predicted by the classifiers

• In the postprocessing we apply some rules to select one

FIRST and one LAST token

Example:

• If more than one token has been predicted as FIRST, take as FIRST the

first token of the negation signal

S A M E S Y S T E M

51

Results

• Baseline: calculating the average length of the scope to the

right of the hedge cue and tagging that number of tokens as scope tokens

• Motivation: 82.45 % of scopes to the right

2.72 2.19 3.15 PCS 3.53 2.26 3.17 PCS-2 Clinical Papers Abstracts BASELINE 27.44 26.21 Clinical 35.92 65.55 PCS 42.37 66.10 PCS-2 Papers Abstracts SYSTEM +36.50 +34.38 Clinical +15.84 +12.02 Papers +12.11 +11.58 Abstracts PCS PCS-2 SYSTEM gold cues

52

Baseline Modality vs Negation

• Baseline results are much lower for the hedge scope finder

53

Results on the abstracts corpus

The system performs clearly better than baseline There is a higher upperbound calculated with gold standard hedge cues

54

Results are lower for papers (PCS -29.63) and clinical (PCS -39.34). Why?

Results on the three corpora

55

Discussion

• Why are the results in papers lower?
• 41 cues (47.00%) in papers are not in abstracts
• Some cues that are in abstracts and are frequent

in papers get low scores.

• Example: suggest

(92.33 PCS in abstracts vs. 62.85 PCS in papers)

56

Discussion

• Errors suggest:
• Bibliographic references
• Sentences with format typical of papers and not
• f abstracts

The conservation from Drosophila to mammals of these two structurally distinct but functionally similar E3 ubiquitin ligases is likely to reflect a combination of evolutionary advantages associated with: (i) specialized expression pattern, as evidenced by the cell-specific expression of the neur gene in sensory organ precursor cells [52]; (ii) specialized function, as suggested by the role of murine MIB in TNF?? signaling [32]; (iii) regulation of protein stability, localization, and/or activity.

57

Discussion

• Why are the results in clinical lower?
• 68 cues (35.45%) in clinical are not in abstracts
• Frequent hedge cues in clinical are not

represented in abstracts

0.00 3.99 21.41

• r

0.00 0.00 0.00 % Abstracts 0.00 3.84 0.00 PCS Clinical 6.67 evaluate for 5.28 consistent with 5.12 rule out %Clinical

CUE

58

Hedge Scope Finder Compared to Negation Scope Finder

• Gold hedge cues = no error

propagation from the first phase

• The abstracts results show

that the same system can be applied to finding the scope of negation and hedge processing

• The systems are equally

portable to the papers corpus

• The negation system is better

portable to the clinical corpus

59

Hedge Scope Finder Compared to Negation Scope Finder

• Error propagation from

the first phase:

• The hedge system is

much less portable to the clinical corpus than the negation system

60

Conclusions

• We have presented a metalearning approach to processing

the scope of negation cues. The metalearner performs better than the object classifiers

• We achieve a 32.07% error reduction over previous

results (Morante et al 2008)

• We have shown that the same scope finding approach can

be applied to both negation and modality

• Finding the scope of modality cues is more difficult
• Modality cues are more diverse and ambiguous than

negation cues

61

Conclusions

• We have shown that the system is portable to different

corpora, although:

• Negation & modality: results are worse for the papers

corpus

• In general, modality cues are less portable across

corpora (Szarvas 2008)

• Negation: results per corpus are mostly determined by

the scores of the negation signals no and not

• Modality: results per corpus are determined by corpus-

specific cues

62

Further Research

• Error analysis to explain:
• why the metalearner performs better than the object

classifiers

• why the papers corpus is more difficult to process
• why some negation signals are more difficult to process

than others

• Experimenting with more features
• dependency syntax
• Test on general domain corpora
• Experimenting with other machine learning approaches

(constraint satisfaction)

63

References

• R. Morante and W. Daelemans. A metalearning approach to

processing the scope of negation. Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL), pages 21–29, Boulder, Colorado, June

• 2009. ACL.
• R. Morante and W. Daelemans. Learning the scope of hedge

cues in biomedical texts. Proceedings of the Workshop on BioNLP, pages 28–36, Boulder, Colorado, June 2009. ACL.

• Roser Morante, Anthony Liekens, and Walter Daelemans.

Learning the Scope of Negation in Biomedical Texts. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 715-724, Honolulu, Hawai, October 2008. ACL.

64

Acknowledgements

• GOA project BIOGRAPH of the

University of Antwerp

• www.biograph.be
• BioScope team
• Thanks for your attention!

65

Results Scope Finding

• Baseline: calculating the average length of the scope to the

right of the negation signal and tagging that number of tokens as scope tokens (85.70 % of scopes to the right)

12.95 4.76 7.11 PCS 62.27 24.86 37.45 PCS-2 80.47 68.11 77.46 F1 76.29 70.86 92.46 IAA 74.96 66.92 78.26 RECALL 86.85 69.34 76.68 PREC Clinical Papers Abstracts BASELINE 71.21 70.75 84.20 82.14 86.38 Clinical +16.74 +16.52 +7.87 +1-.36 +5.27 gold +9.79 +9.26 +13.77 +15.23 +12.26 gold +7.17 +7.29 +8.07 +7.23 +8.92 gold 41.00 66.07 PCS 44.44 66.93 PCS-2 70.94 82.60 F1 69.72 83.45 RECALL 72.21 81.76 PREC Papers Abstracts SYSTEM