Improving Entity Recommendation with Search Log and Multi-Task - - PowerPoint PPT Presentation

Improving Entity Recommendation with Search Log and Multi-Task Learning

Jizhou Huang, Wei Zhang, Yaming Sun, HaifengWang, Ting Liu

• Motivation
• Approach
• Experiment

Outline

Problem

Context-aware entity recommendations are more relevant to a user’s information need

Context-insensitive recommendations

* ⇒ ... ⇒ Chicago

Context-aware recommendations

Dreamgirls ⇒ Chicago

• Context-aware entity recommendation

– Given a query 𝑟#, its context 𝐷# = 𝑟&, 𝑟(, … , 𝑟#*&, and a set of related entities 𝐹# = 𝑓&, 𝑓(, … , 𝑓- , our task is to rank the entities in 𝐹# based on the signals derived from both 𝑟# and 𝐷#

• Examples

Task

𝐷# ⇒ 𝑟# Entity Recommendations Los Angeles travel guide ⇒ Chicago New York City, California, San Francisco, Illinois American rock band ⇒ Chicago The Doobie Brothers, The Beach Boys, Eagles, Cheap Trick Dreamgirls movie trailers ⇒ Chicago Moulin Rouge, Cabaret, The Jazz Singer, Roxie Hart

• Imbalanced entity click logs for ambiguous queries

– Recommendations cannot cover as many intents as possible – Sufficient for the frequently asked meanings of such queries – Insufficient for the rarely asked meanings of such queries

• There may be irrelevant in-session preceding queries

– Not every preceding query addresses the same information need as the current query

Challenges

• Motivation
• Approach
• Experiment

Outline

• Method

– We propose a multi-task DNN model to combine two tasks of entity recommendation (main task) and context-aware ranking (auxiliary task)

• Key intuitions

– The two tasks are closely related in Web search and the representations of input queries and contexts can be naturally shared across them – We can take advantage of the large amounts of search logs in a multi-task learning framework to improve entity recommendation – The clicked documents are helpful in understanding users’ search intents behind a query under variant contexts, which can be beneficial to entity recommendation in a multi-task learning framework

Improved with Search Log and Multi-Task Learning

Multi-Task DNN Model

Query and context representations shared across two tasks Context-aware ranking for Web Search Context-aware entity recommendation BiLSTM di BiLSTM dj el ek cos(vk, vm) cos(vl , vm) P(ek|c, qt) FC layer FC layer cos(vi , vr) cos(vj , vr) P(dj|c, qt) FC layer FC layer FC layer FC layer BiLSTM qt Current query Preceding queries (context, c) BiLSTM q1 BiLSTM q2 BiLSTM qt- ... ... Attention-based weighted average vs Documents Entities vs vi vj vr vm vk vl FC layer Concatenation P(di|c, qt) P(el|c, qt) vq vc

1

• Objective

– Minimize the negative log likelihood of the clicked results −log 2

3,4,56 ∈8

p 𝑠; 𝑑, 𝑟

• Training algorithm

Training

1: Initialize model Θ randomly 2: for 𝑗𝑢𝑓𝑠𝑏𝑢𝑗𝑝𝑜 in 1 ··· 𝑂 do 3: Randomly select a task 𝑈 (context-aware ranking or entity recommendation) 4: Select a random training example for task 𝑈 5: Compute loss for task 𝑈 6: Compute gradient ∇(Θ) 7: Update Θ by taking a gradient step with ∇(Θ) 8: end for

• Motivation
• Approach
• Experiment

Outline

• Ranking

– Use the trained model to compute a score between 𝑑, 𝑟 and 𝑓 ∈ 𝐹 P 𝑓 𝑑, 𝑟 = cos 𝑤N, 𝑤O =

PQRPS ∥PQ∥∥PS∥

• Two ways of using the score to rank entities

– As an individual ranking model – As a feature in a baseline learning to rank framework

Entity Ranking

As an Individual Ranking Model

MBR This method is based on nearest neighbors collaborative filtering proposed by [Fernandez-Tobias and Blanco, 2016] ER This is a context-insensitive DNN model which only considers the current query in generating entity recommendations ER-C This is a single-task DNN model which only uses the entity click logs to train an entity recommendation model ER-C-MT This is the proposed multi-task DNN model

[Fernandez-Tobias and Blanco, 2016] Memory-based recommendations of entities for web search users. 2016. In CIKM.

As a Feature in a Learning to Rank Framework

LTR This baseline is a context-insensitive model comprising a set of non-contextual features for entity recommendation LTR-ER This model is trained with all LTR features and the similarity feature computed by ER LTR-ER-C This model is trained with all LTR features and the similarity feature computed by ER-C LTR-ER-C-MT This model is trained with all LTR features and the similarity feature computed by ER-C-MT

• Training

– Context-aware ranking: 26,426,495 examples of (𝐷U, 𝑟U, 𝐸U)

• 𝐸U = 𝑒U

;, 𝑒X * XY&,…,Z , a clicked doc and 𝐿 randomly-sampled non-clicked docs

– Entity recommendation: 8,821,550 examples of (𝐷

\, 𝑟\, 𝐹 \)

• 𝐹

\ = 𝑓 \ ;, 𝑓] * ]Y&,…,^ , a clicked entity and 𝑀 randomly-sampled non-clicked entities

• Test

– 8,402,881 examples of (𝐷O, 𝑟O, 𝐹O)

• Evaluation metric

– NDCG

Data & Evaluation Metric

As an Individual Ranking Model

• ER-C-MT vs. Baselines

– Highest performance – The most effective in ranking entities for this task

• ER-C-MT and ER-C vs. ER

– Both outperform ER – Preceding queries are useful for improving entity recommendation

• ER-C-MT vs. ER-C

– ER-C-MT outperforms ER-C – Learning the model in a multi- task learning setting can bring further improvements

0.0194 0.0444 0.0641 0.0203 0.0454 0.0663 0.0206 0.0455 0.0675 0.0216 0.0504 0.0710

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08

NDCG@1 NDCG@5 NDCG@10

MBR ER ER-C ER-C-MT

As a Feature in a Learning to Rank Framework

0.1219 0.2103 0.2502 0.1332 0.2261 0.2665 0.1386 0.2324 0.2728 0.1461 0.2438 0.2834

0.05 0.1 0.15 0.2 0.25 0.3

NDCG@1 NDCG@5 NDCG@10

LTR LTR-ER LTR-ER-C LTR-ER-C-MT

• LTR and LTR-ER vs. LTR-

ER-C and LTR-ER-C-MT

– The latter two outperform the former two – Context information can significantly help to improve the performance of entity recommendation

• LTR-ER-C-MT vs. Others

– Highest performance – The performance of entity recommendation can be significantly improved through search logs and multi-task learning

Examples

Query and Context Query: A Song of Ice and Fire Context: Maisie Williams ⇒ Rose Leslie Entity Recommendations LTR: Westworld, Game of Thrones, House of Card, Nip/Tuck, Frozen LTR-ER-C-MT: Isaac Hempstead-Wright, Carice van Houten, Iwan Rheon, Liam

Cunningham, Peter Dinklage

Query and Context Query: Florence Context: Soccer Players ⇒ Roberto Baggio Entity Recommendations LTR: Vatican City, Pompeii, Rome, Metropolitan City of Florence, San Gimignano LTR-ER-C-MT: A.C. Milan, A.S. Roma, Inter Milan, A.C. ChievoVerona, Real Madrid C.F.

• We study the problem of context-aware entity recommendation

– We propose a multi-task DNN model by leveraging Web search logs to improve entity recommendation – We evaluate our approach using large-scale, real-world search logs of a widely used commercial Web search engine

• Our proposed method is effective for this task

– The experiments show that our method significantly outperforms several strong baselines – The experiments also demonstrate that context information can significantly improve the performance of entity recommendation

Conclusions

Thank you!

huangjizhou@baidu.com

百度地图AI算法团队实习生招聘

岗位：算法实习生 方向：数据挖掘、自然语言处理、知识图谱 通过实习，你可以： 1）基于真实的时空大数据对AI算法进行深入研究，发表顶级会议论文 2）了解并实践如何利用百度地图的时空大数据以及百度AIG前沿技术，构建AI驱动的创新产品， 革新用户体验 3）得到具有丰富学术和工程经验的百度技术专家的多对一指导（学术导师+工程实践导师） 工作地点：北京 实习时间：6个月以上 基础要求：有数据挖掘、自然语言处理、知识图谱等任一研究领域经验 感兴趣的同学请发送简历以及求职邮件到：huangjizhou@baidu.com