Dynamic Embeddings for User Profiling in Twitter Shangsong Liang 1 , - - PowerPoint PPT Presentation
Dynamic Embeddings for User Profiling in Twitter Shangsong Liang 1 , Xiangliang Zhang 1 , Zhaochun Ren 2 , Evangelos Kanoulas 3 1 KAUST, Saudi Arabia 2 JD.com, China 3 University of Amsterdam, The Netherlands Overview The Task Background and
1 KAUST, Saudi Arabia 2 JD.com, China 3 University of Amsterdam, The Netherlands
2
Ò The Task Background and Related Work Ò Our Method
Ò Dynamic User and Word Embedding Model (DUWE) Ò Streaming Keyword Diversification Model (SKDM)
Ò Experiments Ò Conclusion
3
Tweets over time Twitter Users Given a user at time t Sport Food
4
Tweets over time Twitter Users Given a user at time t Sport Food
Relevant Diversified Dynamic
5
Ò Expert finding task at TREC 2005 enterprise track
Ò Given documents which describes expert candidates, answer
Ò A generative language modeling approach in Balong et al
Ò Works on a Static document collection Ò Assumes users’ profiling results are unchanged
6
Ò ExperTime (Rybak et al 2014) Ò A probabilistic model for learning how personal research
7
Ò Treat words as atomic units leading to a vocabulary mismatch that
Ò Represent words and users in disjoint vocabulary spaces making
8
Ò Dynamic Topic Models: modeling dynamic user interests
Ò Topic over time model (Wang et al. KDD 2006) Ò Topic tracking model (Iwata et al. IJCAI 2009) Ò Dynamic user clustering topic model (Liang et al. KDD 2016), etc Ò None of them is for user profiling
Ò Dynamic Word Embedding
Ò
Dynamic word embedding by separating data into time bins, and apply word2vec within each bin (Kim et al. 2014, Hamilton et al. 2016)
Ò
Or based on Bayesian skip-gram model (Bamler and Mandt, 2017)
Ò
All of them are for words only but not for users
Ò
All of them are not for user profiling
9
Ò The Task Background and Related Work Ò Our Method
Ò Dynamic User and Word Embedding Model (DUWE) Ò Streaming Keyword Diversification Model (SKDM)
Ò Experiments Ò Conclusion
10
Ò Dynamic User and Word Embedding Model (DUWE)
Ò Infer both users’ and words’ embeddings over time in the
Ò Enable to measure the similarities between users’ and words’
Ò Streaming Keyword Diversification Model
Ò Retrieve relevant keywords to profile users’ current interests
Ò Diversify the returned relevant keywords such that the
11
vt−1 zt−1 βt−1 ut−1 yt−1 αt−1 vt zt βt ut yt αt
n+
t−1
m+
t−1
n+
t
m+
t
V V |Ut| |Ut−1|
Word representation at t-1 User representation at t Observed co-
words at t-1 Observed user-word pairs at t-1
p(Ut | Ut−1) ∝ N(Ut−1,α α α2
t−1I) · N(0,α
α α2
0 I)
p(Vt | Vt−1) ∝ N(Vt−1,β β β2
t−1I) · N(0,β
β β2
0 I)
User Diffusion Word Diffusion
12
According to Kalman filtering, we define the variance of transition kernel for a user embedding from t-1 to t
t−1I) · N(0,α
0 I) Gaussian Prior
measuring the word distribution changes from previous time step t-1 to the current time step t for user u
13
According to Kalman filtering, we define the variance of transition kernel for a word embedding from t-1 to t
Gaussian Prior
measuring the word distribution changes from t-1 to the current time step t
t−1I) · N(0,β
0 I)
14
Ò Apply the skip-gram filtering for the inference (Bamler et al. 2017)
Ò Posterior distribution over and conditional on the statistics
skip-gram model for words skip-gram model for user and words model transition for users model transition for words
positive and negative indicator matrices for all word-to-word pairs positive and negative indicator matrices for all user-to-word pairs
15
Ò generating top-K relevant and diversified keywords for
16
Ò The Task Background and Related Work Ò Our Method
Ò Dynamic User and Word Embedding Model (DUWE) Ò Streaming Keyword Diversification Model (SKDM)
Ò Experiments Ò Conclusion
17
Ò Datasets
Ò 1,375 users randomly sampled from Twitter Ò 3.78 million tweets posted by the users from the beginning of their
Ò Two types of Ground Truth: One for evaluating Relevance-oriented
Ò Evaluation Metrics
Ò Relevance: Pre (Precision), NDCG, MRR, MAP Ò Their semantic version of the metrics, denoted as Pre-S, NDCG-S,
Ò Diversity: Pre-IA (Intent-Aware Precision), α-NDCG, MRR-IA, MAP-IA
18
Ò Baselines
Ò Non-dynamic Embedding Models
Ò
Skip-Gram Model, i.e., word2vec Model (SGM)
Ò
Distributed Representations of Documents (DRD)
Ò Dynamic Traditional Profiling Model
Ò
Predictive Language Model (PLM)
Ò Dynamic Topic Model
Ò
User Clustering Topic model (UCT)
Ò Dynamic Embedding Models
Ò
Dynamic Independent Skip-Gram model (DISG)
Ò
Dynamic Pre-initialized Skip-Gram model (DPSG)
Ò
Dynamic Independent Distributed Representations of documents (DIDR)
Ò
Dynamic Pre-initialized Distributed Representations of documents (DPDR)
19
Ò Average relevance performance on time periods of each
20
Ò Diversity performance on time periods of each month
21
Top-6 keywords of an example user’s dynamic profile, whose interests cover a number of aspects and dramatically change over time, from Sport, fitness, kitchen, exercise, to education.
22
Relevance performance over time Diversity performance over time
23
24
Ò The Task Background and Related Work Ò Our Method
Ò Dynamic User and Word Embedding Model (DUWE) Ò Streaming Keyword Diversification Model (SKDM)
Ò Experiments Ò Conclusion
25
Ò Study the problem of dynamic user profiling in Twitter Ò Propose a Dynamic User and Word Embedding model
Ò Propose a Streaming Keyword Diversification Model
Ò Evaluate the performance of the proposed models in real