U Berlin Institute of Technology Department Machine Learning - - PowerPoint PPT Presentation

U

Berlin Institute of Technology Department Machine Learning

Canonical Trend Analysis for Social Networks

Felix Bießmann, Jens-Michalis Papaioannou, Mikio Braun, Matthias L. Jugel, Klaus-Robert Müller, Andreas Harth

U

Canonical Trends

Temporal Dynamics of Web Data

2

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes)

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005]

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010]

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010] Diffusion of information [Gomez Rodriguez et al, ICML 2011/2012]

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010] Diffusion of information [Gomez Rodriguez et al, ICML 2011/2012]

Canonical Trend Analysis

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010] Diffusion of information [Gomez Rodriguez et al, ICML 2011/2012]

Canonical Trend Analysis

• Exploits temporal structure to find trends

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010] Diffusion of information [Gomez Rodriguez et al, ICML 2011/2012]

Canonical Trend Analysis

• Exploits temporal structure to find trends
• Find web sources that precede/follow trends

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010] Diffusion of information [Gomez Rodriguez et al, ICML 2011/2012]

Canonical Trend Analysis

• Exploits temporal structure to find trends
• Find web sources that precede/follow trends

Examples:

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010] Diffusion of information [Gomez Rodriguez et al, ICML 2011/2012]

Canonical Trend Analysis

• Exploits temporal structure to find trends
• Find web sources that precede/follow trends

Examples:

• Spatiotemporal Dynamics of Retweets to News Articles

U

Canonical Trends

Temporal Dynamics of Web Data

2

Web content is copied, repeated or rephrased (Trends/Memes) This temporal structure contains important information Growing interest in temporal dynamics of graphs

Understanding dynamic graphs [Leskovec et al, KDD, 2005] Causal Inference [Lozano and Sindhwani, NIPS 2010] Diffusion of information [Gomez Rodriguez et al, ICML 2011/2012]

Canonical Trend Analysis

• Exploits temporal structure to find trends
• Find web sources that precede/follow trends

Examples:

• Spatiotemporal Dynamics of Retweets to News Articles
• Music trends on Last.fm

U

Canonical Trends

Canonical Correlation Analysis

3

Z

Latent Variable (Trend)

Y

Features

(e.g. Bag of Words, User actions, edge histograms, ...)

X

U

Canonical Trends

Canonical Correlation Analysis

3

Z

Latent Variable (Trend)

Y

Features

(e.g. Bag of Words, User actions, edge histograms, ...)

X

w>

x X

w>

y Y

U

Canonical Trends

Canonical Correlation Analysis

3

Z

Latent Variable (Trend)

Y

Features

(e.g. Bag of Words, User actions, edge histograms, ...)

X

[Jordan 1875], [Hotelling 1936], [Bach and Jordan 2006]

argmax

wx, wy

w>

x XY >wy

q w>

x XX>wxw> y Y Y >wy

w>

x X

w>

y Y

U

Canonical Trends

Canonical Trend Model

4

Z

Latent Variable (Trend)

Y

Features

(e.g. Bag of Words, User actions, edge histograms, ...)

X

faster

U

Canonical Trends

Canonical Trend Model

4

Z

Latent Variable (Trend)

Y

Features

(e.g. Bag of Words, User actions, edge histograms, ...)

X

faster

w>

y Yt

X

τ

wx(τ)>Xtτ

U

Canonical Trends

An Example on News Trends

5

U

Canonical Trends

An Example on News Trends

5

mashable.com arstechnica.com techcrunch.com slashdot.org

U

Canonical Trends

An Example on News Trends

5

mashable.com arstechnica.com techcrunch.com slashdot.org mashable.com arstechnica.com techcrunch.com slashdot.org

Time t=-1 t=0

U

Canonical Trends

An Example on News Trends

5

mashable.com arstechnica.com techcrunch.com slashdot.org mashable.com arstechnica.com techcrunch.com slashdot.org

Time t=-1 t=0

Xf ∈ RW ×T

U

Canonical Trends

An Example on News Trends

5

mashable.com arstechnica.com techcrunch.com slashdot.org mashable.com arstechnica.com techcrunch.com slashdot.org

Time t=-1 t=0

Y = X

f 06=f

Xf 0 Xf ∈ RW ×T

U

Canonical Trends

An Example on News Trends

5

mashable.com arstechnica.com techcrunch.com slashdot.org mashable.com arstechnica.com techcrunch.com slashdot.org

Time t=-1 t=0

Predict future content

• f all other web sources

from past content of single web source

Y = X

f 06=f

Xf 0 Xf ∈ RW ×T

U

Canonical Trends

An Example on News Trends

5

mashable.com arstechnica.com techcrunch.com slashdot.org mashable.com arstechnica.com techcrunch.com slashdot.org

Time t=-1 t=0

Y = X

f 06=f

Xf 0 Xf ∈ RW ×T X

τ

wx(τ)>Xtτ w>

y Yt

Z

U

Canonical Trends

Why Projecting to Canonical Subspace?

6

Easily interpretable: For Text data each canonical direction is a topic

[De Bie and Cristianini, 2004]

Information theoretic optimal compression

[Creutzig 2009]

Conversion of canonical correlations to granger causality index

[Otter 1991]

U

Canonical Trends

Canonical Trend Analysis For Social Networks

7

U

Canonical Trends

Canonical Trend Analysis For Social Networks

7

Quantifying spatiotemporal retweet response to news content

U

Canonical Trends

Canonical Trend Analysis For Social Networks

7

Quantifying spatiotemporal retweet response to news content Finding users ahead and following music trends on Last.fm

U

Canonical Trends

Canonical Trend Analysis For Social Networks

8

U

Canonical Trends

Canonical Trend Analysis For Social Networks

8

Some news web site publishes some content ...

U

Canonical Trends

Canonical Trend Analysis For Social Networks

8

Some news web site publishes some content ... Time

t

... which is retweeted

t + τ1

U

Canonical Trends

Canonical Trend Analysis For Social Networks

8

Some news web site publishes some content ... Time

t

... which is retweeted

t + τ1

... at different locations

U

Canonical Trends

Data Extraction

9

U

Canonical Trends

Data Extraction

9

f ∈ {1, 2, . . . , F}

For each news site extract

U

Canonical Trends

Data Extraction

9

f ∈ {1, 2, . . . , F}

For each news site extract Bag-of-Words Features

Xf = [xf(t = 1), . . . , xf(t = T)] ∈ RW ×T

U

Canonical Trends

Data Extraction

9

f ∈ {1, 2, . . . , F}

For each news site extract Retweet locations

Yf = [yf(t = 1), . . . , yf(t = T)] ∈ RL×T

Bag-of-Words Features

Xf = [xf(t = 1), . . . , xf(t = T)] ∈ RW ×T

U

Canonical Trends

Data Extraction: Retweet Locations

10

U

Canonical Trends

Data Extraction: Retweet Locations

10

• 1. Extract URI of each news article in twitter stream

U

Canonical Trends

Data Extraction: Retweet Locations

10

• 1. Extract URI of each news article in twitter stream
• 2. Retrieve Location from Twitter User Profile

U

Canonical Trends

Data Extraction: Retweet Locations

10

• 1. Extract URI of each news article in twitter stream
• 2. Retrieve Location from Twitter User Profile
• 3. Resolve Ambiguities / Remove non-sense Locations

U

Canonical Trends

Data Extraction: Retweet Locations

10

• 1. Extract URI of each news article in twitter stream
• 2. Retrieve Location from Twitter User Profile
• 3. Resolve Ambiguities / Remove non-sense Locations
• 4. Downsample Geographic Locations

U

Canonical Trends

Mean Locations of Retweeted News Articles

11

U

Canonical Trends

Downsampling of Geographic Information

12

GADM: An RDF spatial representation

• f all the administrative

regions in the world

U

Canonical Trends

Canonical Trend Analysis

13

News Content (Bag-of-Words) Retweet Locations

Z

Hidden Variable (News Topic)

ˆ yf(t) = X

τ

wy(τ)>Yf(:, t + τ) ˆ xf(t) = w>

x Xf(:, t)

U

Canonical Trends

Canonical Trend Analysis

14

Optimal and

argmax

wy(τ),wx

Corr(ˆ xf(t), ˆ yf(t)). ˆ xf(t) = w>

x Xf(:, t)

ˆ yf(t) = X

τ

wy(τ)>Yf(:, t + τ)

News Content (Bag-of-Words) Retweet Locations

wx ∈ RW wy(τ) ∈ RW Nτ

U

Canonical Trends

Efficient Computation of Canonical Trends

15

(linear) ‘Kernel Trick’

• Very efficient for high-dimensional feature spaces

˜ Yf = 2 6 4 Yf,τ=1 . . . Yf,τ=Nτ 3 7 5 ∈ RLNτ ⇥T .

[Takens 1981]

U

Canonical Trends

Efficient Computation of Canonical Trends

15

Temporal Embedding

• Standard CCA problem

(linear) ‘Kernel Trick’

• Very efficient for high-dimensional feature spaces

[Jordan 1875], [Hotelling 1936], [Anderson 1999] [Fyfe 2000], [Fukumizu 2007]

˜ Yf = 2 6 4 Yf,τ=1 . . . Yf,τ=Nτ 3 7 5 ∈ RLNτ ⇥T .

wy(τ) = Yf,τα, wx = Xfβ.

[Takens 1981]

U

Canonical Trends

Efficient Computation of Canonical Trends

16

Objective function is maximized in the dual where are linear kernels

K ˜

Y = ˜

Y > ˜ Y KX =X>X

Corr(ˆ x(t), ˆ y(t)) = P

τ(wy(τ)>Yτ)>Xwx

pP

τ(wy(τ)>YτY > τ wy(τ))w> x XX>wx

= α>K ˜

Y KXβ

q α>K2

˜ Y αβ>K2 Xβ

U

Canonical Trends

Efficient Computation of Canonical Trends

17

Dual coefficients are solution to generalized eigenvalue equation

 K ˜

Y KX

KXK ˜

Y

 α β

• = λ

 K2

˜ Y + Iκy

K2

X + I + κx

 α β

• Corr(ˆ

x(t), ˆ y(t)) = P

τ(wy(τ)>Yτ)>Xwx

pP

τ(wy(τ)>YτY > τ wy(τ))w> x XX>wx

= α>K ˜

Y KXβ

q α>K2

˜ Y αβ>K2 Xβ

U

Canonical Trends

Efficient Computation of Canonical Trends

17

Bießmann et al, Machine Learning, 2010

Dual coefficients are solution to generalized eigenvalue equation

 K ˜

Y KX

KXK ˜

Y

 α β

• = λ

 K2

˜ Y + Iκy

K2

X + I + κx

 α β

• Corr(ˆ

x(t), ˆ y(t)) = P

τ(wy(τ)>Yτ)>Xwx

pP

τ(wy(τ)>YτY > τ wy(τ))w> x XX>wx

= α>K ˜

Y KXβ

q α>K2

˜ Y αβ>K2 Xβ

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

18

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

18

Mean PCA Canonical Trends

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

18

Mean PCA Canonical Trends

argmax

wy(τ),wx

Corr(ˆ xf(t), ˆ yf(t)).

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

18

Mean PCA

w>

x = 1x/N, wy(τ) = 1y/N

Canonical Trends

argmax

wy(τ),wx

Corr(ˆ xf(t), ˆ yf(t)).

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

18

Mean PCA

argmax

wy(τ)

(wy(τ)> ˜ Yf ˜ Y >

f wy(τ)),

argmax

wx

(w>

x XX>wx),

s.t. wy(τ)>wy(τ) = w>

x wx = 1

w>

x = 1x/N, wy(τ) = 1y/N

Canonical Trends

argmax

wy(τ),wx

Corr(ˆ xf(t), ˆ yf(t)).

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

18

Mean PCA Canonical Trends Hypothesis News Content helps predicting retweet frequency Mean Wordcount predicts mean tweet frequency best Wordcount variance predicts tweet variance

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

18

Mean PCA Canonical Trends

U

Canonical Trends

Comparisons: Mean, PCA and Canonical Trends

19

U

Canonical Trends

Canonical Convolution

20

Excerpts from LA Times Spatiotemporal Response California New York Ontario

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute a Bag-of-Word subspace (topic) and

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute a Bag-of-Word subspace (topic) and spatiotemporal twitter response patterns

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute a Bag-of-Word subspace (topic) and spatiotemporal twitter response patterns such that news content and retweets are maximally correlated

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute a Bag-of-Word subspace (topic) and spatiotemporal twitter response patterns such that news content and retweets are maximally correlated Results can be interpreted w.r.t

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute a Bag-of-Word subspace (topic) and spatiotemporal twitter response patterns such that news content and retweets are maximally correlated Results can be interpreted w.r.t

• How much impact has a news site on Twitter-Community

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute a Bag-of-Word subspace (topic) and spatiotemporal twitter response patterns such that news content and retweets are maximally correlated Results can be interpreted w.r.t

• How much impact has a news site on Twitter-Community
• (Content that will lead to high retweet frequency)

U

Canonical Trends

Spatiotemporal Analysis of Retweets of News

21

We use canonical correlation analysis to compute a Bag-of-Word subspace (topic) and spatiotemporal twitter response patterns such that news content and retweets are maximally correlated Results can be interpreted w.r.t

• How much impact has a news site on Twitter-Community
• (Content that will lead to high retweet frequency)
• (Where and when maximal impact is reached)

U

Canonical Trends

Users and Trends on Last.fm

22

A last.fm user subgraph

18-Nov-2007 25-Nov-2007 02-Dec-2007 09-Dec-2007 lfm:user/JoanLandor#i _:b1 #weeklychart _:b2 #weeklychart lfm:user/popnutten foaf:knows #from #to _:a1 #list New Order #rank_1 Tiger Lou #rank_2 Aereogramme #rank_3 new wave #tag electronic #tag indie #tag alternative #tag #from #to _:a2 #list Air #rank_1 Death Cab for Cutie #rank_2 The Notwist #rank_3 #tag #tag #tag

U

Canonical Trends

Users and Trends on Last.fm

23

Xf = [xf(t = 1), . . . , xf(t = T)] ∈ RM×T Yf = X

f 06=f

Xf 0 Extract Weekly Chartlist Last.fm-Music-tags Single User Chart Time Series All Other Users

U

Canonical Trends

Users and Trends on Last.fm

23

Canonical Correlogram ρ(τ) = Corr

• wx(τ)>Xτ, w>

y Y

• =

wx(τ)>XτY >wy wx(τ)>XτX>

τ wx(τ) · w> y Y Y >wy

= α>KτKY β α>K2

τ α · β>K2 Y β

Xf = [xf(t = 1), . . . , xf(t = T)] ∈ RM×T Yf = X

f 06=f

Xf 0 Extract Weekly Chartlist Last.fm-Music-tags Single User Chart Time Series All Other Users

U

Canonical Trends

Users and Trends on Last.fm

24

U

Canonical Trends

Users and Trends on Last.fm

24

Time lag (weeks) Top Tags −10 −5 5 10 deutscher hip h german hip hop german reggae minimal synth minimalist −10 −5 5 10 0.1 0.2 0.3 0.4 0.5 Time lag (weeks) Correlation 8 10 12 14 16 18

Behind the Trend

U

Canonical Trends

Users and Trends on Last.fm

24

Time lag (weeks) Top Tags −10 −5 5 10 punk cabaret crust punk hardcore punk punk blues emo −10 −5 5 10 0.2 0.3 0.4 0.5 Time lag (weeks) Correlation 5 10 15 20

Ahead of Trend

Time lag (weeks) Top Tags −10 −5 5 10 deutscher hip h german hip hop german reggae minimal synth minimalist −10 −5 5 10 0.1 0.2 0.3 0.4 0.5 Time lag (weeks) Correlation 8 10 12 14 16 18

Behind the Trend

U

Canonical Trends

Summary

25

U

Canonical Trends

Summary

25

Canonical Trend Analysis (CTA)

U

Canonical Trends

Summary

25

Canonical Trend Analysis (CTA)

Finds maximally correlated subspace of graph feature time series

U

Canonical Trends

Summary

25

Canonical Trend Analysis (CTA)

Finds maximally correlated subspace of graph feature time series Efficient computations via representer theorem

U

Canonical Trends

Summary

25

Canonical Trend Analysis (CTA)

Finds maximally correlated subspace of graph feature time series Efficient computations via representer theorem CTA between news content and retweet location

U

Canonical Trends

Summary

25

Canonical Trend Analysis (CTA)

Finds maximally correlated subspace of graph feature time series Efficient computations via representer theorem CTA between news content and retweet location Reveals ‘strongest’ topics and spatiotemporal tweet response

U

Canonical Trends

Summary

25

Canonical Trend Analysis (CTA)

Finds maximally correlated subspace of graph feature time series Efficient computations via representer theorem CTA between news content and retweet location Reveals ‘strongest’ topics and spatiotemporal tweet response CTA between users on Last.fm

U

Canonical Trends

Summary

25

Canonical Trend Analysis (CTA)

Finds maximally correlated subspace of graph feature time series Efficient computations via representer theorem CTA between news content and retweet location Reveals ‘strongest’ topics and spatiotemporal tweet response CTA between users on Last.fm Finds users ahead and behind musical trends

U

Canonical Trends

Future Work

26

U

Canonical Trends

Future Work

26

Sparse, non-negative canonical directions

U

Canonical Trends

Future Work

26

Sparse, non-negative canonical directions Other features than BoW

U

Canonical Trends

Future Work

26

Sparse, non-negative canonical directions Other features than BoW Online optimization

U

Canonical Trends

Future Work

26

Sparse, non-negative canonical directions Other features than BoW Online optimization What about Nonstationarities?

U

Canonical Trends

Detecting ‘Trendsetting’ News Websites

27

Real Data Example:

BoW Features from 96 Technology News Feeds in October 2011

yf(t) ˆ yf(t)

U

Canonical Trends

Comparison Canonical Trend Analysis and LSA

28

Canonical Topics predict overall topics better than Latent Semantic Indexing Canonical trend analysis between and

• vs. LSA on and separately

Xf Xf Yf Yf