Kristina T outanova Joint work with entities relations city_of - PowerPoint PPT Presentation

Kristina T outanova Joint work with

entities relations city_of Honolulu born_in United States Barack Obama spouse Michelle Obama

city_of Honolulu born_in lived_in United States Barack Obama nationality spouse Michelle Obama

city_of Honolulu born_in United States Barack Obama nationality spouse nationality Michelle Obama

city_of Chicago Honolulu born_in Barack Obama worked in lived_in United States Barack Obama Chicago. spouse A photo of Barack Obama ’s Michelle Obama Chicago house. Goal: use both sources of information. [Lao et al. 2012], [Gardner et al. 2013,2014] [Riedel et al. 2013] [Neelakantan et al 2015]

Embedding-based models for KB completion Using text: Universal Schema •

-0.1 Encoding relevant properties of the 2.3 entities, predictive of their relationships. -1.4 Michelle Obama Encoding relevant properties of the -0.7 relations that help define the set of lived_in -3.4 entity pairs for which the relation holds. 1.6 lived_in Michelle Chicago f(Michelle Obama, lived_in, Chicago) RESCAL [Nickel et al 2011], TransE [Bordes et al 2011, 2013], Bilinear-diagonal [Yang et al. 2015]

city_of Honolulu Chicago born_in lived_in United States Barack Obama spouse founder Michelle Obama Textual relations KB relations

Embedding-based models for KB completion Using text: Universal Schema • Sparsity of textual relations • Compositional representations of text

Basic KB relations Textual relations Conv KB relations Textual relations

[Toutanova, Chen, Pantel, Poon, Choudhury, Gamon, EMNLP 2015]

Evaluation on held out queries : Where did Michelle Obama live? Mean reciprocal rank of first correct answer (times 100) 41 40.1 40 KB Inference 39 37.7 38 KB + text-basic 37.3 37 KB+text 36 compositional 35 Mean Reciprocal Rank (MRR) FB15K-237 new dataset from Freebase and ClueWeb. http://research.microsoft.com/en-us/downloads/3a9bf02d-b791-4e95-b88d-389feef3e421/

Embedding-based models for KB completion Using text: Universal Schema • Are node/relation embeddings sufficient • Using observed features

spouse ? Michelle Obama Honolulu born_in Barack Obama PEOPLE_BORN_IN ? Honolulu spouse Michelle Obama

f(x=Michelle Obama, spouse , y=Barack Obama) = 𝑥 1 𝜚 1 = 99.8 Honolulu born_in spouse y x Barack Obama spouse Michelle Obama

100 82.2 80 56.2 E+Bilinear-diag 60 40 Simple Observed 20 0 MRR

Embedding-based models for KB completion Using text: Universal Schema • Inference from multi-step relation paths from KB and text • Efficient compositional representation and learning

IL2 MAPK1 [Toutanova, Lin, Yih, Poon, Quirk, ACL 2016]

IL2 MAPK1

𝐺 𝑚 𝑡, 𝑢 = ෍ 𝑄 𝑢 𝑡, 𝜌 Φ 𝜌 ) 𝜌 =𝑚 𝐺 𝑚+1 𝑡, 𝑢 = ෍ 𝐺 1 𝑡, 𝑙 𝐺 𝑚 (𝑙, 𝑢) 𝑙

Hits@10 on Gene Regulation 52.53 54 52 48.6 50 48.27 48 46 44 42 39.92 40 38 36 Hits@10 Bilinear-diag NBestPaths All Paths All Paths+Nodes