Building Natural Language System based on Theoretical Linguistics - PowerPoint PPT Presentation

Building Natural Language System based on Theoretical Linguistics 理論言語学に基づいた自然言語処理システム @MiCS 2019/10/23 Masashi Yoshikawa (NAIST D3)

Univ.), and now back in Nara Self Introduction @Kuwait 2012 • NAIST Matsumoto-ken D3 • Like: syntactic/semantic parsing, structured prediction • Originally from Osaka Univ. (Foreign Studies) • mainly worked on Turkish and Arabic languages • Spent 2.5 years of my Ph.D period at Bekki-sensei’s lab (Ochanomizu • Surprised to know everyone is working on IE at the lab (no more parsing) 2

(Ice Breaker?) Arabic Morphology is mi XY a Z u KTB write ... X a Y a Z a did X aa Y i Z u doer ya XY a Z u do ma XY a Z a place to do Three Concept Consonant times Syntactic Template ma XY uu Z u is patient to X a YY a Z a made one do X a Y ii Z u adjective ... Three consonants representing concepts Syntactic Templates seek sink ɣRB new DRS study QRʔ read ʔKL eat JDD QRR decide ðHB go carry QLL few QBL accept SJD head down QʕD sit deciding syntactic function Ħ ML Ṭ LB 3

(Ice Breaker?) Arabic Morphology is ... mosque few deciding syntactic function Syntactic Templates representing concepts Three consonants adjective ma DR a S a X a Y ii Z u made one do X a YY a Z a is patient to ma XY uu Z u place to do basement mi Qʕ a D Three Concept Consonant times Syntactic Template K aa T i B u Fill XYZ wrote K a T a B a book K i T aa B u writer west ma SJ i D u school pregnant student decide Q a RR a R a ma ɣR i D u Q a L ii L u mi XY a Z u ma XY a Z a do QRR QLL carry go ðHB new decide JDD QBL eat ʔKL read QRʔ study DRS ya XY a Z u few accept seek doer X aa Y i Z u did X a Y a Z a ... write KTB with ABC SJD sink ɣRB sit QʕD head down Ħ ML Ṭ LB Ṭ aa L i B u Ħ aa M i L u 3

(Ice Breaker?) Arabic Morphology is ma DR a S a student decide Q a RR a R a ma ɣR i D u Q a L ii L u ma SJ i D u mi Qʕ a D basement school mosque few deciding syntactic function Syntactic Templates representing concepts Three consonants ... Three Concept Consonant times Syntactic Template pregnant west made one do PROPN ADV VERB PROPN ADP PRON ADJ NOUN ADP VERB K aa T i B u are adequate for these languages? with ABC Fill XYZ wrote K a T a B a book K i T aa B u writer X a Y ii Z u adjective X a YY a Z a new SJD is patient to QBL few QLL carry go ðHB decide QʕD QRR JDD eat ʔKL read QRʔ study DRS head down accept sit X a Y a Z a ma XY uu Z u place to do mi XY a Z u ma XY a Z a do ya XY a Z u doer X aa Y i Z u did ɣRB Taro went to the new school in which Hanako studies as well ... write KTB seek sink Ħ ML Ṭ LB Ṭ aa L i B u Ħ aa M i L u • Sematic languages (Hebrew, Amharic..) • Implication: recent subword methods اضيا اهيف وكاناه سردت يتلا ةديدجلا ةسردلنا ىلا ورات بهذ • But its syntax is familiar to us • VSO with postpositional modifiers 3

太郎 は 学校 へ 行っ た What is Syntactic Theory? Taro okula gitti ... • Provide explanations for phenomena arising from the way words are concatenated • PP-attachment: "John ( saw a girl ( with a telescope )) " • Coordination: "Wendy ( ran 19 miles ) and ( walked 9 miles ) " • control verb, complement, passive/active voice, scope, etc. • Must be general to cover all languages, while describing language specificities • e.g. Universal Dependencies (de Merneffe et al., 2014) Taro went to school ةسردلنا ىلا ورات بهذ 4

Combinatory Categorial Grammar Steedman 2000, Bekki 2010 argument NP ( S \ NP )/ NP S \ NP S \ NP ( S \ NP )/( S \ NP ) NP NP / N N S return value N • Categories with recursive function-like structure X / Y • A small number of derivational rules (less than 10) X \ Y • Meta rules (cf. CFG: S �-? NP VP ) • Forward/backward application: X �-? X / Y Y X �-? Y X \ Y • Forward/backward composition rules: X / Z �-? X / Y Y / Z a man is beating John 5

Combinatory Categorial Grammar N ( S \ NP )/ NP return value argument NP ( S \ NP )/ NP S \ NP S \ NP ( S \ NP )/( S \ NP ) NP NP / N Steedman 2000, Bekki 2010 S N NP • Categories with recursive function-like structure X / Y • A small number of derivational rules (less than 10) X \ Y • Meta rules (cf. CFG: S �-? NP VP ) • Forward/backward application: X �-? X / Y Y X �-? Y X \ Y • Forward/backward composition rules: X / Z �-? X / Y Y / Z a man is beating John 5

Combinatory Categorial Grammar NP / N ( S \ NP )/ NP NP ( S \ NP )/ NP return value argument NP ( S \ NP )/ NP S \ NP S \ NP ( S \ NP )/( S \ NP ) NP N Steedman 2000, Bekki 2010 N S \ NP S • Categories with recursive function-like structure X / Y • A small number of derivational rules (less than 10) X \ Y • Meta rules (cf. CFG: S �-? NP VP ) • Forward/backward application: X �-? X / Y Y X �-? Y X \ Y • Forward/backward composition rules: X / Z �-? X / Y Y / Z a man is beating John 5

Combinatory Categorial Grammar NP S \ NP S \ NP ( S \ NP )/ NP NP ( S \ NP )/ NP return value argument NP ( S \ NP )/ NP S \ NP S \ NP ( S \ NP )/( S \ NP ) NP / N N N ( S \ NP )/( S \ NP ) S Steedman 2000, Bekki 2010 • Categories with recursive function-like structure X / Y • A small number of derivational rules (less than 10) X \ Y • Meta rules (cf. CFG: S �-? NP VP ) • Forward/backward application: X �-? X / Y Y X �-? Y X \ Y • Forward/backward composition rules: X / Z �-? X / Y Y / Z a man is beating John 5

Combinatory Categorial Grammar ( S \ NP )/( S \ NP ) N ( S \ NP )/( S \ NP ) S \ NP S \ NP ( S \ NP )/ NP NP ( S \ NP )/ NP return value argument NP ( S \ NP )/ NP S \ NP S \ NP NP NP / N S N NP / N Steedman 2000, Bekki 2010 N • Categories with recursive function-like structure X / Y • A small number of derivational rules (less than 10) X \ Y • Meta rules (cf. CFG: S �-? NP VP ) • Forward/backward application: X �-? X / Y Y X �-? Y X \ Y • Forward/backward composition rules: X / Z �-? X / Y Y / Z a man is beating John 5

Combinatory Categorial Grammar S \ NP NP / N NP / N N ( S \ NP )/( S \ NP ) S \ NP S \ NP ( S \ NP )/ NP NP ( S \ NP )/ NP return value argument NP ( S \ NP )/ NP S \ NP ( S \ NP )/( S \ NP ) NP S N NP Steedman 2000, Bekki 2010 N NP / N • Categories with recursive function-like structure X / Y • A small number of derivational rules (less than 10) X \ Y • Meta rules (cf. CFG: S �-? NP VP ) • Forward/backward application: X �-? X / Y Y X �-? Y X \ Y • Forward/backward composition rules: X / Z �-? X / Y Y / Z a man is beating John 5

Combinatory Categorial Grammar S \ NP NP argument return value ( S \ NP )/ NP NP ( S \ NP )/ NP S \ NP S \ NP ( S \ NP )/( S \ NP ) N NP / N NP / N NP S \ NP ( S \ NP )/ NP S \ NP ( S \ NP )/( S \ NP ) Steedman 2000, Bekki 2010 N S S N NP / N NP • Categories with recursive function-like structure X / Y • A small number of derivational rules (less than 10) X \ Y • Meta rules (cf. CFG: S �-? NP VP ) • Forward/backward application: X �-? X / Y Y X �-? Y X \ Y • Forward/backward composition rules: X / Z �-? X / Y Y / Z a man is beating John 5

Basic CCG-based Semantic Parsing argument 0 is john and ... Dictionary S NP NP S\NP (S\NP)/NP (word, category) to a lambda term based on event semantics language (e.g., Haskell) • Imagine functional programming \x y �-? f(x,y) : lambda term john, mary: entity term true, false: truth term • Hand-crafted dictionary maps Mary John likes • F : NP �=? F • Here we use logical formulas • F : N �=? \x �-? F(x) • F : (S\NP)/NP �=? \y x �-? exist e. F(e) ��../ e • There exists an event , whose • F : S\NP �=? \x �-? exist e. F(e) & A0(0) ��../ 6

Basic CCG-based Semantic Parsing NP language (e.g., Haskell) (word, category) to a lambda term based on event semantics Dictionary argument 0 is john and ... S NP S\NP (S\NP)/NP • Imagine functional programming \x y �-? f(x,y) : lambda term john, mary: entity term true, false: truth term \y x �-? exist e. like e & A0 x & A1 y mary • Hand-crafted dictionary maps Mary John likes • F : NP �=? F • Here we use logical formulas • F : N �=? \x �-? F(x) • F : (S\NP)/NP �=? \y x �-? exist e. F(e) ��../ e • There exists an event , whose • F : S\NP �=? \x �-? exist e. F(e) & A0(0) ��../ 6

Basic CCG-based Semantic Parsing argument 0 is john and ... Dictionary S NP (S\NP)/NP S\NP NP (word, category) to a lambda term based on event semantics language (e.g., Haskell) • Imagine functional programming \x y �-? f(x,y) : lambda term \x �-? exist e. like e & A0 x & A1 mary john, mary: entity term true, false: truth term \y x �-? exist e. like e & A0 x & A1 y mary • Hand-crafted dictionary maps Mary John likes • F : NP �=? F • Here we use logical formulas • F : N �=? \x �-? F(x) • F : (S\NP)/NP �=? \y x �-? exist e. F(e) ��../ e • There exists an event , whose • F : S\NP �=? \x �-? exist e. F(e) & A0(0) ��../ 6