LanguageIndependent LanguageIndependent AnswerPrediction - PowerPoint PPT Presentation

Language�Independent Language�Independent Answer�Prediction Answer�Prediction from�the�Web from�the�Web Alejandro�Figueroa�&�Gü ünter�Neumann nter�Neumann Alejandro�Figueroa�&�G Language�Technology�Laboratory Language�Technology�Laboratory German�Research�Center�for�Artificial�Intelligence� German�Research�Center�for�Artificial�Intelligence� DFKI,�Saarbr Saarbrü ücken cken DFKI,�

Motivation:�From From Search Search Engines Engines to� to�Answer Answer Motivation:� Engines Engines User�Query: KeyWrds,�Wh/Clause,�Q/Text Search Engines Experienc ed/based Interactiv e QA� cycles Answer Engine User�still�carries the Shift more „interpretation major�efforts in� effort“ to�machines understanding

e.g., AskMRS, Mulder, NSIR, SmartWeb, … Web/ /based�Question�Answering:�SOA based�Question�Answering:�SOA Web Exact answ 1 NL-Question Exact answ 2 … Q-type, a-type, NL-query NL-Question Analysis Answer Extraction NL-Query Snippets Query Refinement IR-Query Document processing Fetched Search Engine documents Problematic: NL-question-driven Query Refinement Usually large set of snippets/documents have to be consulted

Web/ /based�Question�Answering:�Our�Approach based�Question�Answering:�Our�Approach Web Exact answ 1 Exact answ 2 … NL-Question Q-type, a-type, NL-query NL-Query Analyse Answer Extraction NL-string Document Snippets processing Search Engine Fetched Answer documents Prediction AP-phrases

Answer�prediction�... Answer�prediction�... • • … identifies�and�extracts�those�substrings�(called�the�AP … identifies�and�extracts�those�substrings�(called�the�AP/ /phrases)� phrases)� from�a�set�of�snippets/text�fragments from�a�set�of�snippets/text�fragments – Which�are�either�paraphrases�of�substrings�of�the�NL�question,�o Which�are�either�paraphrases�of�substrings�of�the�NL�question,�or r – – Which�contain�exact�answer�strings – Which�contain�exact�answer�strings – E.g E.g.,� .,�„Who is the primer�minister of�Great�Britain ?“ – • Possible • Possible AP AP/ /phrases phrases:� :�„United�Kingdom“,�„ „Tony�Blair“ • • Notice: Notice: – Often,�the�first�N Often,�the�first�N/ /snippets�do�not�contain�the�answer�and�even�point�to� snippets�do�not�contain�the�answer�and�even�point�to� – answerless�documents�(we�cannot�rely�on�the�first�3/ /5� 5� answerless�documents�(we�cannot�rely�on�the�first�3 snippets/documents;�~�low�MRR�of�snippets) snippets/documents;�~�low�MRR�of�snippets) – We�need�to�consider�several�snippets/documents�in�order�to�make� We�need�to�consider�several�snippets/documents�in�order�to�make�use� use� – of�redundancy�(only�checking�the�first�is�not�enough�even�if�it�contains� contains� of�redundancy�(only�checking�the�first�is�not�enough�even�if�it� the�answer) the�answer) – Snippets�are�usually�not�very�much�linguistically�well – Snippets�are�usually�not�very�much�linguistically�well/ /formed formed – They�are�computed�online�with�very�fast,�but�cheap�methods They�are�computed�online�with�very�fast,�but�cheap�methods –

Where�is�the�tal conference�series�taking�place�in�2006�?

Technological�Roadmap�for�this�work Technological�Roadmap�for�this�work Data/ /driven driven Data Language�independent� Language�independent� No�initial�query�expansion/ No�initial�query�expansion/ answer�prediction answer�prediction refinement�without� refinement�without� / No�NLP�components / No�NLP�components initial�documents initial�documents / No�language�model No�language�model / Unsupervised� Unsupervised� data/ /management management data / No�parameter��smoothing / No�parameter��smoothing / No�restrictions�on� No�restrictions�on� / length�of�AP length�of�AP/ /phrases� phrases� / No�fixed�window�size No�fixed�window�size / How�far�can�we�go�with�this�?

Answer�Prediction�as�text�zooming:�core�steps Answer�Prediction�as�text�zooming:�core�steps N/snippets�+ NL/question Document Construction: Token set W Sentence set S Ranking of sentences Extraction of AP-phrases Ranking of AP-phrases Extraction of exact Shallow�Answer�extraction:� Shallow�Answer�extraction:� answers answers Only�relevant�so�far�for�performing� Only�relevant�so�far�for�performing� evaluation�of��answer�prediction evaluation�of��answer�prediction

Document�Construction/Representation Document�Construction/Representation • Sentences�from�snippets� • A� • • Sentences�from�snippets� A�document document D�is�represented�by� D�is�represented�by� &�NL�question &�NL�question the�following�elements: the�following�elements: – Only�very�simple/local� Only�very�simple/local� – tokenizer &�sentence� &�sentence� tokenizer = < ω ω ε ω ω ε > D { , , , freq ( , , ) , markers markers i j i j ∀ ε i , j , , ω ω ε > freq ( , , ) 0 , • Collect�global�statistics • i j Collect�global�statistics ≤ ε ≤ Υ 0 }  1 if the word w is in S at position k . i s  = X sik Length of�the longest Sentence in�D  0 Otherwise . – Word�pair Word�pair/ /distance� distance� – All�possible�units�|W|x|W|xY W|x|W|xY,� ,� |W|� All�possible�units�| frequency�(respecting� |W|� frequency�(respecting� relative�small�(N/ /snippets) snippets) relative�small�(N order) order) len ( S ) σ ∑ ∑ s ω ω ε = freq ( , , ) X X i j si ( k − ε ) sjk = = ε + s 1 k 1

Ranking�of�Sentences Ranking�of�Sentences • A� • Filtering:�ignore�low� • • A�matrix matrix for for each each sentence sentence: : Filtering:�ignore�low� frequent�elements�(ζ ζ=2� =2� frequent�elements�( globally). globally).  ω ω ε < freq ( , , ) if i j i j  ⇒  = ω ω ε > M ( S ) freq ( , , ) if j i ∀ ≤ ζ = i , j M M 0 ij s j i ij ij   0 otherwise To�avoid bias of�long�sequences of�low correlated words • rank(Ss • rank(Ss) ) is is given given by by the the maximal� maximal�eigenvalue eigenvalue of�M: of�M: = λ rank ( Ss ) ( M ( S )) max s This�eigenvalue gives�the�amount�of�„ syntactic�bonding� force “ captured�by�the�eigenvector�related�to� λ max .

Ranking�Sentences Sentences / / Remarks Remarks Ranking� • Retrieval�of�snippets�is�biased�by�terms�in�the�query • Retrieval�of�snippets�is�biased�by�terms�in�the�query • Snippets� not�only� consists� of�query� terms,� they� also� • Snippets� not�only� consists� of�query� terms,� they� also� consists�of�enriched�contextual�information consists�of�enriched�contextual�information • Our� ranking� schema� identifies� • Our� ranking� schema� identifies� strong� syntactic� strong� syntactic� patterns in�snippets�using�them�to�rank�the�sentences� in�snippets�using�them�to�rank�the�sentences� patterns in�the�snippets� in�the�snippets� • A�high�ranked�sentence� • A�high�ranked�sentence�not not necessary�contains�query� necessary�contains�query� terms,�but�might�contain�the�answer terms,�but�might�contain�the�answer • What�is�the�difference�between�this�approach�and�a� • What�is�the�difference�between�this�approach�and�a� ranking�based�on� n n/ /grams grams (e.g.,� (e.g.,�AskMRS AskMRS) ) ranking�based�on� – We�do�not�have�any�dependency�on� We�do�not�have�any�dependency�on�lengths lengths – – We�do�not�need�to�estimate� We�do�not�need�to�estimate�back back/ /off off probabilities probabilities – – We�do�not�have�the�problem�that� We�do�not�have�the�problem�that�long�sentences long�sentences will�tend�to� will�tend�to� – have�a�lower�rank lower�rank than� than�small�sentences small�sentences have�a�