Expediting Search Trend Detection via Prediction of Query Counts - - PowerPoint PPT Presentation
Expediting Search Trend Detection via Prediction of Query Counts Nadav Golbandi Liran Katzir* Yehuda Koren* Ronny Lempel Yahoo! Labs, Haifa, Microsoft Research, Google, Haifa, Israel Yahoo! Labs, Haifa, Israel ATL-Israel Israel
Expediting Search Trend Detection via Prediction of Query Counts
Nadav Golbandi
Yahoo! Labs, Haifa, Israel
Liran Katzir*
Microsoft Research, ATL-Israel
Yehuda Koren*
Google, Haifa, Israel
Ronny Lempel
Yahoo! Labs, Haifa, Israel
International Conference on Web Search and Data Mining, February 6th - 8th, 2013 Rome, Italy
Search Trends
Search queries with a sudden popularity surge
0.2 0.4 0.6 0.8 1 12 24 36 48 60
#queries Time in Hours
Amanda Peet 0.2 0.4 0.6 0.8 1 12 24 36 48 60
#queries Time in Hours
Hybrid Cars
Trending Topics
Microsoft MSN biggest movers Yahoo! Trending Now Google Trends
Prior-art Detection Algorithm
1. Create a language model at time . 2. Estimate trendiness score
, = log | −max
3. Output top-k queries.
lowers the score of periodic queries
Proposed Method:
New Algorithm
query counts during [t-1,t) Prediction Procedure predicted counts for [t,t+1) Prior-art Algorithm trending searches for time t+1 query counts during [t-1,t) Prior-art Algorithm trending searches for time t
Goal: Keep precision and recall, reduce detection time !
New Algorithm
query counts during [t-1,t) Prediction Procedure predicted counts for [t,t+1) Prior-art Algorithm trending searches for time t+1 query counts during [t-1,t) Prior-art Algorithm trending searches for time t
Intuition – counts time series
() () () !() Query q’s count … time
1m 1m 1m 1m
Intuition – prediction input
#(, 30)
() () () !() Query q’s count … time
Intuition – prediction target
#(, 30)
() () () !() Query q’s count … time
31, 32 … 90 What will be the query volume during following hour?
…
Intuition – prediction target value
#(, 30)
() () () !() Query q’s count … time
31, 32 … 90 What will be the query volume during following hour?
…
& , 30 =
+ + ⋯ + !
#(, ′) & *(, ′)
Explanatory and Response variables
, , … , ! + + ⋯ + ! #(, 30) &(, 30) #(, ) &(, ) #(, 31) &(, 31) #(, 32) &(, 32)
…
?
#(, ′) & *(, ′)
Explanatory and Response variables
, , … , ! + + ⋯ + ! #(, 30) &(, 30) #(, ) &(, ) #(, 31) &(, 31) #(, 32) &(, 32)
…
? Training Phase
(only on trends)
Runtime Phase
(on all queries)
Prediction Models
Auto-regressive Time-smoothed, single model
This method fits a vector - such that
.
Prediction Models
ℎ
.
Auto-regressive Time-smoothed, per-hour model
This method fits a vector -(ℎ)for every hour ℎ such that
.
Prediction Models
60 62 64 66 68 70 72 15 30 45 60 75 90 Per-Hour Model Single Model
% of good predictions Length of #(, )
ℎ
.
Auto-regressive Time-smoothed, per-hour model
This method fits a vector -(ℎ)for every hour ℎ such that
.
Example of a specific hour model
2 4 6 8 10
Coefficient
Index
Proposed Method
New Algorithm
query counts during [t-1,t) Prediction Procedure predicted counts for [t,t+1) Prior-art Algorithm trending searches for time t+1 query counts during [t-1,t) Prior-art Algorithm trending searches for time t
Experimental Precision and Recall
0.95 0.97 0.99 1.01 1.03 1.05 10 20 %improvement Precision 0.95 0.97 0.99 1.01 1.03 1.05 10 20 %improvement Recall Number of search trends Number of search trends
Experimental detection time
10 20 30 5 10 15 20 Number of search trends Detection time Average reduction in detection time
An alternative perspective
Building a language model in a temporal corpus involves a trade off. Taking a long history creates bias. Taking a short history creates variance. The new method can be viewed as a time- sensitive language model.
Conclusion/Discussion
language model as a building block in a temporal dynamic environment.
performance (precision and recall), with reduction in detection time.