Introduction to Natural Language Processing MORPHOLOGY - - PowerPoint PPT Presentation
Introduction to Natural Language Processing MORPHOLOGY TRANSDUCERS Martin Rajman Martin.Rajman@epfl.ch and Jean-C edric Chappelier Jean-Cedric.Chappelier@epfl.ch Artificial Intelligence Laboratory M. Rajman LIA
✬ ✫ ✩ ✪ Introduction to Natural Language Processing
MORPHOLOGY – TRANSDUCERS
Martin Rajman
Martin.Rajman@epfl.ch
and Jean-C´ edric Chappelier
Jean-Cedric.Chappelier@epfl.ch
Artificial Intelligence Laboratory
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 1/24
✬ ✫ ✩ ✪ Objectives of this lecture ➥ Present morphology, important part of NLP ➥ Introduce transducers, tools for computational morphology
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 2/24
✬ ✫ ✩ ✪ Contents ➥ Morphology ➥ Transducers ➥ Operations and Regular Expressions on Transducers
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 3/24
✬ ✫ ✩ ✪
Morphology Study of the internal structure and the variability of the words in a language:
✏ verbs conjugation ✏ plurals ✏ nominalization (enjoy → enjoyment) ➜ inflectional morphology: preserves the grammatical category
give given gave gives ...
➜ derivational morphology: change in category
process processing processable processor processabilty
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 4/24
✬ ✫ ✩ ✪
Morphology (2) Interest: use a priori knowledge about word structure to decompose it into morphemes and produce additional syntactic and semantic information (on the current word) processable → process-
☞ 2 morphemes
meaning: process possible role: root suffix semantic information: main less The importance and complexity of morphology vary from language to language Some information represented at the morphological level in English may be represented differently in other languages (and vice-versa). The paradigmatic/syntagmatic repartition changes from one language to another Example in Chinese: ate −
→ expressed as ”eat yesterday”
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 5/24
✬ ✫ ✩ ✪
Stems – Affixes Words are decomposed into morphemes: roots (or stems) and affixes. There are several kinds of affixes:
➊ prefixes:
in- -credible
➋ suffixes:
incred- -ible
➌ infixes:
Example in Tagalog ( Philippines): hingi (to borrow) → humingi (agent of the action) In slang English! → ”fucking” in the middle of a word Man-fucking-hattan
➍ circumfixes:
Example in German: sagen (to say) → gesagt (said)
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 6/24
✬ ✫ ✩ ✪
Stems – Affixes (2) several affixes may be combined: examples in Turkish where you can have up to 10 (!) affixes. uygarlas ¸tıramadıklarimizdanmıs ¸sınızcasına uygar las ¸ tır ama dık lar imiz dan mıs ¸ sınız casına civilized +BEC +CAUS +NEGABLE +PPART +PL +P1PL +ABL +PAST +2PL +ASIF as if you are among those whom we could not cause to become civilized When only prefixes and suffixes are involved: concatenative morphology Some languages are not concatenative:
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 7/24
✬ ✫ ✩ ✪
Example of semitic languages Pattern-based morphology In Hebrew, the verb morphology is based on the association of
Example: LMD (learn or teach) LAMAD → he was learning LUMAD → he was taught
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 8/24
✬ ✫ ✩ ✪
Computational Morphology Let us consider flexional morphology, for instance for verbs and nouns Noun flexions: plural General rule: +s but several exceptions (e.g. foxes, mice) Verb flexions: conjugations
☞ How to handle flexions (comptutationaly)?
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 9/24
✬ ✫ ✩ ✪
Computational Morphology Example:surface form: is canonical representation at the lexicon level (formalization): be+3+s+Ind+Pres The objective of computational morphology tools is precisely to go from one to the other:
Challenge: have a ”good” implementation of these two transformations Tools: associations of strings → transducers
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 10/24
✬ ✫ ✩ ✪
String associations
(X1, X′
1)
. . .
(Xn, X′
n)
(eaten, eat) (processed, process) . . . (thought, think) Easy situation: ∀i,
|Xi| = |X′
i|
Example: (abc, ABC)
⇒ represented as a sequence of character transductions
(abc, ABC) = (a,A)(b,B)(c,C)
☞strings on a new alphabet: strings of character couples
Not so easy: If ∃i,
|Xi| = |X′
i| ⇒ requires the introduction of empty string ε
Example: (ab, ABC) ≃ (εab, ABC) = (ε,A)(a,B)(b,C)
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 11/24
✬ ✫ ✩ ✪
Dealing with ε Where to put the ε? Example:(ab,ABC) ≃ (εab, ABC) but also (ab,ABC) ≃ (aεb, ABC)
General case:
n m (with m < n)
Hard problem in general → need for a convention
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 12/24
✬ ✫ ✩ ✪
Transducer (definition) Let Σ1 and Σ2 be two enumerable sets (alphabets), and
Σ =
A transducer is a DFSA on Σ Σ1 : ”left” language
: upper language : input language
Σ2 : ”right” language
: lower language : output language
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 13/24
✬ ✫ ✩ ✪
Example 1 2 initial state final state(s)
b a:b a b:a b:a b:ε a
Some transductions: (bb,b) [0,0,2] (ababb,baab) [0,1,2,0,0,2]
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 14/24
✬ ✫ ✩ ✪
Different usages of a transducer
➊ association checking
(abba, baaa)∈ Σ∗ ?
➋ Generation: string1 → string2 bbab → ? ➌ Analysis: string2 → string1
? → ba
➊: easy: (= FSA: nothing special)
What about ➋ and ➌ ?
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 15/24
✬ ✫ ✩ ✪
Transduction Walk through the FSA following one or the other element of the couple (projections)
The fact that a transducer is a deterministic (couple-)FSA does not at all imply that the automaton resulting from one projection or the other is also deterministic! non-deterministic evaluation backtracking on ”wrong” solutions
⇒ The projection is not constant time (in general)
When a transducer is deterministic with respect to one projection or the other, it is called a sequential transducer A transducer in not sequential in general. In particular if one language or the other (upper
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 16/24
✬ ✫ ✩ ✪
Transduction (2) Example: bbab → ?
b:b b:b
1
a:b
2
b:a
2
b:ε a:a b:b
2
b:ε
2
b:ε
1
b:b
2
a:b
1
b:a bbab → bbba bbab → ba
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 17/24
✬ ✫ ✩ ✪
Transduction (3) Example:? → ba
b:b
2
b:ε a:a
2
b:ε
1
a:a
(FAIL)
1
a:b
2
a:a
2
b:a
2
b:ε
(FAIL)
aa → ba ab → ba bbab → ba
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 18/24
✬ ✫ ✩ ✪
Operations and Regular Expressions
➮ All FSA regular expressions: concatenation, or, Kleene closure (*), ...
Example:(concatenation) ”a:b c:a” recognizes ac and produces ba
➮ cross-product of regular languages: E1 ⊗ E2 recognizes L1 × L2
example: a+ ⊗ b+ → (an, bm)
∀ n ≥ 1, m ≥ 1
!! this is = (a ⊗ b)+
➮ Composition of transducers: T = T1 ◦ T2 (X1, X2) ∈ T ⇐ ⇒ ∃Y : (X1, Y ) ∈ T1 and (Y, X2) ∈ T2 ➮ Reduction: extraction of the upper or the lower FSA
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 19/24
✬ ✫ ✩ ✪
(Other) examples of applications (morphology)
★ text-to-speech (grapheme to phoneme transduction) ★ specific lexicon representation (composition of some access and inverse fonctions) ★ filters (remove/add/modify marks; e.g. HTML) ★ text segmentation
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 20/24
✬ ✫ ✩ ✪
Computational morphology using transducers Use of composition:
➛ Identification of a paradigm (T1) ➛ Implementation of this paradim (T2) ➛ Exception handling (T3)
Example: input: chat+NP , fox+NP , ... (”+NP” means ”noun plural”)
T1: ([a-z]+)(\+NP ⊗ \+1)
paradigm identification: plural nouns (trivial here:
T2: ([a-z]+)(\+1 ⊗ \+Xs)
plural inflection of nouns (regular part)
T3: ([a-z]+)(h\+Xs ⊗ hes | x\+Xs ⊗ xen | ... | [ˆhx...](\+X⊗ε)s) correction of exceptions T1 ◦ T2 ◦ T3: plural for nouns
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 21/24
✬ ✫ ✩ ✪
Computational morphology using transducers (2) Detailed example on the plural of nouns: general case: add a terminal ’s’ cat+NP → cats, dog+NP → dogs, ... Exceptions (several kind):
Method: find all the paradigms (linguists’ role) and implement a transducer for each of them
☞ add the paradigm identification in the lexical description
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 22/24
✬ ✫ ✩ ✪
Keypoints
➟ Flexional and derivational morphologies, their roles ➟ Main functions of transducers: association checking, generation and analysis ➟ Deterministic and not deterministic nature of transduction
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 23/24
✬ ✫ ✩ ✪
References
Book, 1997.
LIA I&C Introduction to Natural Language Processing (CS-431)
J.-C. Chappelier 24/24
Computational Linguistics Martin Rajman Artificial Intelligence Laboratory
Add “s” to the end of the singular form
(dog, dogs) (arrow, arrows) ...
Some “regular” plurals need to be modified to be easy to pronounce (“euphonic rules)
“es” instead of “s”
(guess, guesses) (box, boxes) (buzz, buzzes) (catch, catches) (dish, dishes)
... but (systematic exception) if the final “ch” is pronounced “k”, add “s” instead of “es”
(stomach, stomachs)
as well as some fully irregular exceptions
(ox, oxen)
“y”, change the “y” to “ies”
(baby, babies) (fly, flies)
Note: there must be a consonant before the “y”...
(boy, boys) (buy, buys)
(hair, ---) (mud, ---)
... but the regular plurals may also be acceptable in specific contexts: “Her hair is black” ... but ... “I saw at least one grey hair, and there are probably
more grey hairs there” (hair, hairs) “They throw mud at each other” ... but ... “These subterranean muds are being removed” (mud, muds)
to the plural “Deer have antlers”
Note that there is a (subtle) difference for a noun not to have a plural (i.e. to be uncountable), or to have a plural form that is the same as the singular one uncountable: “Her hair is black” is correct, while “Her hair are black” is not invariable: “This deer is fast” and “Deer are fast” are both correct (but do not mean the same)
“fe” to “ves”
(half, halves) (knife, knives)
... but
(belief, beliefs) (if, ifs) “There are so many ifs and buts in this policy"
(crisis, crises) (hypothesis, hypotheses)
... but
(vis, vires) where “vis” is a Latin word meaning “power” that has been imported in English, while preserving its Latin plural (“vires”) “An example of vis is the influence of the leader"
(tomato, tomatoes) (mosquito, mosquitoes) (volcano, volcanoes)
... but
(photo, photos) (video, videos) (piano, pianos)
much more complicated modification
(man, men) (mouse, mice) (foot, feet) (tooth, teeth) ...
Computational Linguistics Martin Rajman Artificial Intelligence Laboratory
and
words
cats, book, flies, ...
cat+N+p, book+N+s, fly+N+p, ...
The typical format of a canonical representations is: Lemma+GrammaticalCategory+MorphoSyntacticFeature1+MorphoSyntacticFeature2+... where:
dictionaries, e.g. the singular form for nouns, or the infinitive for verbs;
category of the word, e.g. N for a noun, Adj for an adjective, or V for a verb;
morphosyntactic features (e.g. the number, the gender, the tense, the person, etc.) that are relevant to identify a specific inflection of a word; and
to the surface form "cats" expresses in a formal way that "cats" is the flection of the noun "cat" corresponding to its plural form ("p" being the tag for the value "plural" of the morphosyntactic feature "number");
representation "turn+V+Ind+Pres+3+s" to the surface form "turns" expresses in a formal way that the surface form corresponding to the flection of the verb ("V") "to turn" at the 3rd person ("3") singular ("s") of the present ("Pres") indicative ("Ind") is "turns".
Implementing some Computational Morphology for English nouns is finding an efficient way of representing a, potentially very large, set of (canonical representation, surface form) associations, such as:
(cat+N+s, cat) (cat+N+p, cats) (book+N+s, book) (book+N+p, books) (fly+N+s, fly) (fly+N+p, flies) (fox+N+s, fox) (fox+N+p, foxes) (deer+N+s, deer) (deer+N+p, deer) (mouse+N+s, mouse) (mouse+N+p, mice) (ox+N+s, ox) (ox+N+p, oxen) ... By "efficient way", we mean a method that:
having to write them explicitly one-by-one;
algorithmic complexity able to produce the surface form(s) associated with a given canonical representation ("generation"),
the canonical representation(s) associated with a given surface form ("analysis")
The idea is to use the composition T1 o T2 o T3 of 3 transducers:
systematic transformation rule(s) to be implemented for regular forms
rules
corresponds to a single systematic rule
numbered here as rule 1
the form “root+N+p”, where root is any possible nominal root, to the intermediate string “root+1”: T1 = ([a-z]+)((\+N\+p)x(\+1)) where “x” represents the “cross-product” operator, “\” is a special character that prevents the character “+” to be interpreted as the Kleene plus operator, and “[a-z]” represents any alphabetic character
When applied to the list
( cat+N+p, book+N+p, fly+N+p, fox+N+p, deer+N+p, mouse+N+p,
T1 represents the following list of associations (cat+N+p, cat+1) (book+N+p, book+1) (fly+N+p, fly+1) (fox+N+p, fox+1) (deer+N+p, deer+1) (mouse+N+p, mouse+1) (ox+N+p, ox+1)
Add “s” to the end of the root (as, for nouns, the root corresponds to the singular form)
form “root+1” to a new intermediate string of the form “rootXs”, where the character X (called the “trace”) identifies the “border” between the root and the suffix “s”: T2 = ([a-z]+)((\+1)x(Xs)) Note: placing a trace X in the new intermediate string will make it easier to handle the various exceptions to be implemented in T3
When applied to the list (resulting from T1)
( cat+1, book+1, fly+1, fox+1, deer+1, mouse+1,
T2 represents the following list of associations (cat+1, catXs) (book+1, bookXs) (fly+1, flyXs) (fox+1, foxXs) (deer+1, deerXs) (mouse+1, mouseXs) (ox+1, oxXs)
If the root ends in “x”, change the ending “x” to “xes”
If the root ends in “y”, change the ending “y” to “ies”
“rootxXs” (resp. “rootyXs”) to a new intermediate string of the form “rootxes” (resp. “rooties”), where “rootx” (resp. “rooty”) is any root ending in “x” (resp. “y”) : T3 = ([a-z]+)(((xXs)x(xes))|((yXs)x(ies))|([^xy]((Xs)x(s)))) where “[^xy]” represents any character but “x” or “y”
When applied to the list (resulting from T2)
( catXs, bookXs, flyXs, foxXs, deerXs, mouseXs,
T3 represents the following list of associations (catXs, cats) (bookXs, books) (flyXs, flies) (foxXs, foxes) (deerXs, deers) (mouseXs, mouses) (oxXs, oxes)
When applied to the original list
(cat+N+p, book+N+p, fly+N+p, fox+N+p, deer+N+p, mouse+N+p,
T1 o T2 o T3 represents the following list of associations (cat+N+p, cats) (book+N+p, books) (fly+N+p, flies) (fox+N+p, foxes) (deer+N+p, deers) (mouse+N+p, mouses) (ox+N+p, oxes) where the first 4 associations are correct, but the last 3 (in red) are erroneous and would require a more sophisticated definition of the transducer T3 responsible for handling the exceptions