Towards finer-grained tagging of discourse connectives Yannick - - PowerPoint PPT Presentation

References

Towards finer-grained tagging of discourse connectives

Yannick Versley SFB 833 / Univ. Tübingen AG Beyond Semantics, DGfS 2011

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Example

◮ I regret this , but the vote has already been taken and [❛r❣✶the

decision is made] [❝♦♥♥so] [❛r❣✷let us leave the matter there] . Ich bedauere das , aber die Abstimmung ist durchgeführt worden , [❛r❣✶die Entscheidung ist gefallen] , [❝♦♥♥also] [❛r❣✷lassen wir die Dinge] . [’Contingency.Cause.Result’]

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Finding discourse relations

Discourse connectives (explicit relations)

◮ co-/subordinating conjunctions (but,after) ◮ discourse adverbials (however, then) ◮ Discontinuous units

(if . . . then, either . . . or)

◮ (Some) can be modified

3 weeks after he left, mainly because she likes strawberries Syntactic structure

◮ purpose clauses ◮ embedded V1

Unmarked relations (implicit relations)

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Explicit vs. Implicit

◮ Many connectives are unambiguous

(e.g. because, although)

◮ About 46% of discourse relations are implicit ◮ Low performance on implicit relations

(Pitler et al., 2009; Lin et al., 2009)

◮ Unsupervised learning on unambiguous connectives

does not help classification of implicits (Marcu and Echihabi, 2002; Sporleder and Lascarides, 2008)

⇒ What about ambiguous connectives?

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Previous work

◮ Use connectives, syntactic configuration to predict lower

levels of the discourse structure (Soricut and Marcu, 2003).

◮ Some connectives are ambiguous, but tense is a good

disambiguator (Miltsakaki et al., 2005).

◮ Some connectives are ambiguous, but shallow features help

(Haddow, 2005).

◮ Most connectives are not ambiguous, and syntax/structure

helps with the rest (Pitler and Nenkova, 2009).

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Corpora with Discourse Structure

RST

◮ Relations between arbitrary segments ◮ Relation inventory: 119 Relations (core inventory of ≈ 20)

PDTB I

◮ Only explicit relations

Inter-sentential links only for discourse adverbials

◮ No relation labels

PDTB II

◮ Taxonomic schema of discourse relations

Level 1: Contingency, Comparison, Expansion, Temporal Level 2: 12 relations (Contrast/Concession, Cause/Consequence) Level 3: ≈30 relations

◮ implicit relations, AltLex, NoRel, EntRel

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Experiment on ambiguous connectives

Take explicit relations, with features:

◮ The connective ◮ Connective at start of sentence? ◮ Structural information (Pitler and Nenkova, 2009)

◮ parent, sibling nodes

with semantic function labels (-TMP, -PRP)

◮ VP in right sibling ◮ trace in right sibling

◮ Tense information on Arg2 (Miltsakaki et al., 2005)

◮ be-form ◮ have-form ◮ main verb form Yannick Versley Towards finer-grained tagging of discourse connectives

References

Problematic connectives (1): coarse

connective frequency conn c+first c+synB c+synA c+verb(arg1) since 154 0.571 0.571 0.675 0.935 0.909 finally 30 0.633 0.933 0.867 0.867 0.933 in turn 27 0.704 0.704 0.704 0.704 0.704 even as 11 0.727 0.636 0.364 0.455 0.636 while 652 0.729 0.727 0.729 0.839 0.805 as 588 0.786 0.786 0.781 0.810 0.781 as long as 20 0.800 0.786 0.750 0.700 0.750

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Problematic connectives (2): fine

connective freq conn c+first c+synB c+synA c+arg1 c+synB+arg1 in fact 70 0.300 0.386 0.286 0.300 0.343 0.429 although 277 0.498 0.588 0.520 0.549 0.592 0.606 still 156 0.500 0.429 0.462 0.506 0.417 0.449 since 154 0.571 0.571 0.669 0.929 0.903 0.896 though 187 0.588 0.652 0.540 0.551 0.652 0.652 while 652 0.598 0.598 0.604 0.718 0.667 0.672 indeed 86 0.605 0.593 0.593 0.593 0.570 0.558 when 837 0.611 0.608 0.609 0.609 0.596 0.588 yet 88 0.648 0.648 0.523 0.432 0.523 0.545 as 588 0.745 0.745 0.736 0.767 0.743 0.745 but 2767 0.790 0.790 0.789 0.788 0.789 0.785 meanwhile 160 0.800 0.800 0.800 0.775 0.794 0.794

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Is there a problem?

◮ Many connectives are unambiguous on the coarse level, but

ambiguous at the finer levels

◮ The features that automatic parsing does not give us

(semantic function labels, traces) would be useful

◮ Tense information helps (for some connectives)

Can we improve the classification part?

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Hierarchical classification

Idea: use information on different levels of the taxonomy Flat classification:

◮ Only consider lowest level of the taxonomy

Greedy classification:

◮ First classify coarse relation ◮ Then classify finer relations

Hierarchical classification

◮ Used by Ciaramita et al. (2003) for WSD

Coarse level: supersenses Fine level: actual synsets for words

◮ Here: relations on different levels

(Contingency, Contingency.cause, C.cause.result)

Yannick Versley Towards finer-grained tagging of discourse connectives

References

How well does it work? (1)

flat conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.954 0.954 0.954 0.953 0.952 0.952 d=2 0.847 0.847 0.845 0.845 d=3 0.796 0.798 hierarchical conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.954 0.953 0.954 0.953 0.952 0.952 d=2 0.847 0.847 0.845 0.845 d=3 0.796 0.798

Yannick Versley Towards finer-grained tagging of discourse connectives

References

How well does it work? (2)

flat conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.954 0.954 0.954 0.953 0.952 0.952 d=2 0.847 0.847 0.845 0.845 d=3 0.796 0.798 greedy conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.955 0.954 0.955 0.953 0.953 0.953 d=2 0.847 0.847 0.845 0.845 d=3 0.798 0.800

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Better features?

An Apples-vs-Oranges comparison: while während always Contrast 0.662 0.711 tense+. . . 0.619 0.622 +wordpairs+productions 0.638 0.784

(Accuracy on first relation)

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Summary

◮ More connectives ambiguous on lower taxonomy levels ◮ Tense and syntactic structure address different problems ◮ Reconstruct semantic functions, traces? ◮ Limited improvements from fancier classification

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Thanks for listening!

THE END

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Ciaramita, M., Hofmann, T., and Johnson, M. (2003). Hierarchical semantic classification: Word sense disambiguation with world

• knowledge. In 18th International Joint Conference on Artificial

Intelligence (IJCAI 2003). Haddow, B. (2005). Acquiring a disambiguation model for discourse connectives. Master’s thesis, School of Informatics, University of Edinburgh. Lin, Z., Kan, M.-Y., and Ng, H. T. (2009). Recognizing implicit discourse relations in the Penn Discourse Treebank. In EMNLP 2009. Marcu, D. and Echihabi, A. (2002). An unsupervised approach to recognizing discourse relations. In ACL 2002. Miltsakaki, E., Dinesh, N., Prasad, R., Joshi, A., and Webber, B. (2005). Experiments on sense annotations and sense disambiguation of discourse connectives. In TLT 2005.

Yannick Versley Towards finer-grained tagging of discourse connectives

References

Pitler, E., Louis, A., and Nenkova, A. (2009). Automatic sense prediction for implicit discourse relations in text. In ACL-IJCNLP 2009. Pitler, E. and Nenkova, A. (2009). Using syntax to disambiguate explicit discourse connectives in text. In ACL 2009 short papers. Soricut, R. and Marcu, D. (2003). Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference (HLT/NAACL-2003). Sporleder, C. and Lascarides, A. (2008). Using automatically labelled examples to classify rhetorical relations: An

• assessment. Natural Language Engineering, 14(3):369–416.

Yannick Versley Towards finer-grained tagging of discourse connectives