http://cs224w.stanford.edu 10/31/2012 Jure Leskovec, Stanford - - PowerPoint PPT Presentation

CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University

http://cs224w.stanford.edu

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 2

We will analyze the following model:

 Nodes arrive in order 1,2,3, … , 𝑜  When node 𝑘 is created it makes a

single out-link to an earlier node 𝑗 chosen:

• 1) With prob. 𝑞, 𝑘 links to 𝑗 chosen uniformly at

random (from among all earlier nodes)

• 2) With prob. 1 − 𝑞, node 𝑘 chooses node 𝑗

uniformly at random and links to a node i points to.

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 3

[Mitzenmacher, ‘03]

Node i

p − + = 1 1 1 α

CLAIM: the model generates networks with power-law degree distribution with exponent:

 Plan: Analyze 𝒆𝒋(𝒖): continuous deterministic

in-degree of node 𝑗 at time 𝑢 > 𝑗

 Initial condition:

• 𝑒𝑗(𝑢) = 0, when 𝑢 = 𝑗 (node i just arrived)

 Expected change of 𝒆𝒋(𝒖) over time:

• With prob. 𝑞 node 𝑢 + 1 links randomly:
• Links to our node 𝑗 with prob. 1/𝑢
• With prob. 1 − 𝑞 node 𝑢 + 1 links preferentially:
• Links to our node 𝑗 with prob.

𝑒𝑗(𝑢) 𝑢

𝒆𝒋 𝒖 + 𝟐 − 𝒆𝒋 𝒖 = 𝐪 𝟐 𝒖 + 𝟐 − 𝒒 𝒆𝒋(𝒖) 𝒖

 How does 𝒆𝒋(𝒖) change as 𝒖→∞?

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 4

Node i

 Expected change of 𝒆𝒋 𝒖 :

• 𝒆𝒋(𝒖 + 𝟐) − 𝒆𝒋(𝒖) = 𝒒

𝟐 𝒖 + 𝟐 − 𝒒 𝒆𝒋(𝒖) 𝒖

• d𝑒𝑗(𝑢)

d𝑢

= 𝑞

1 𝑢 + 1 − 𝑞 𝑒𝑗(𝑢) 𝑢

=

𝑞+𝑟𝑒𝑗(𝑢) 𝑢

• 1

𝑞+𝑟𝑒𝑗(𝑢) d𝑒𝑗(𝑢) = 1 𝑢 d𝑢

• ∫

1 𝑞+𝑟𝑒𝑗(𝑢) d𝑒𝑗(𝑢) = ∫ 1 𝑢 d𝑢

• 1

𝑟 ln 𝑞 + 𝑟𝑒𝑗 𝑢

= ln 𝑢 + 𝑑

• 𝑞 + 𝑟𝑒𝑗 𝑢 = 𝐵 𝑢𝑟 ⇒ 𝒆𝒋 𝒖 =

𝟐 𝒓 𝑩𝒖𝒓 − 𝒒

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 5

𝑟 = (1 − 𝑞) integrate Let 𝐵 = 𝑓𝑑 and exponentiate Divide by 𝑞 + 𝑟 𝑒𝑗(𝑢)

A=?

What is the value of constant A?

 We know: 𝑒𝑗 𝑗 = 0  So: 𝑒𝑗 𝑗 = 1

𝑟 𝐵𝑗𝑟 − 𝑞 = 0

 ⇒ 𝑩 = 𝒒

𝒋𝒓

 And so ⇒ 𝒆𝒋 𝒖 = 𝒒

𝒓 𝒖 𝒋 𝒓

− 𝟐

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 6

𝒆𝒋 𝒖 = 𝟐 𝒓 𝑩𝒖𝒓 − 𝒒

Note: Old nodes (small 𝑗 values) have higher in-degrees 𝑒𝑗(𝑢)

 What is 𝑮(𝒍) the fraction of nodes that has

degree at least 𝒍 at time 𝒖?

• How many nodes i have degree > 𝒍?
• 𝑒𝑗 𝑢 =

𝑞 𝑟 𝑢 𝑗 𝑟

− 1 > 𝑙

• Solve for 𝑗 and obtain: 𝐣 < 𝐮

𝒓 𝒒 𝒍 − 𝟐 −𝟐

𝒓

 There are 𝒖 nodes total at time 𝒖 so the

faction 𝑮(𝒍) is:

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 7

q

k p q k F

1

1 ) (

−

      + =

Note: F(k) is a CCDF

• f the degree

distribution

 What is the fraction of nodes with

degree exactly 𝒍?

• Take the derivative of −𝐺(𝑙) w.r.t 𝑙
• 𝐺(𝑙) is CCDF, so −𝐺𝐺(𝑙) is the PDF

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 8

p k p q p k F

q

− + = ⇒       + =

− −

1 1 1 1 1 ) ( '

1 1

α

q.e.d.

q

k p q k F

1

1 ) (

−

      + =

 Pref. attachment gives power-law degrees  Intuitively a reasonable process  Can tune 𝑞 to get the observed exponent

• On the web, 𝑄[𝑜𝑜𝑒𝑓 ℎ𝑏𝑏 𝑒𝑓𝑒𝑒𝑓𝑓 𝑙] ~ 𝑙−2.1
• 2.1 = 1 + 1/(1 − 𝑞) ⇒ 𝒒 ~ 𝟏. 𝟐

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 9

 Two changes from the Gnp

• (1) Growth
• (2) Preferential attachment

 Do we need both? Yes!

• Add growth to Gnp (i.e., 𝑞 = 1):
• 𝑦𝑘 = degree of node 𝑘 at the end
• 𝑌

𝑘(𝑣) = 1 if 𝑣 links to 𝑘, else 0

• 𝑌

𝑘 = 𝑌 𝑘(𝑘 + 1) + 𝑌 𝑘(𝑘 + 2) + ⋯ + 𝑌 𝑘(𝑜)

• 𝐹[𝑌

𝑘(𝑣)] = 𝑄[𝑣 𝑚𝑗𝑜𝑙𝑏 𝑢𝑜 𝑘] = 1/(𝑣 − 1)

• 𝐹 𝑌

𝑘 = ∑ 1 𝑣−1 𝑜 𝑘+1

= 1

𝑘 + 1 𝑘+1 + ⋯ + 1 𝑜−1 = 𝐼𝑜−1 – 𝐼 𝑘

• 𝐹[𝑌

𝑘] = log

(𝑜 − 1) – log (𝑘) = log ((𝑜 − 1)/𝑘) NOT 𝑜

𝑘 𝛽

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 10

Hn…nth harmonic number: 𝐼𝑜 = 1 𝑙 ≈ log (𝑜)

𝑜 𝑙=1

 Preferential attachment is not so good at

predicting network structure

• Age-degree correlation
• Solution: Node fitness (virtual degree)
• Links among high degree nodes
• On the web nodes sometime avoid linking to each other

 Further questions:

• What is a reasonable model for how people

sample through web-pages and link to them?

• Short random walks
• Effect of search engines – reaching pages based on

number of links to them

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 11

𝒆𝒋 𝒖 = 𝒒 𝒓 𝒖 𝒋

𝒓

− 𝟐

3 3 3 2 2 log

log log log ) 1 log( log log

> = < < =              =

−

α α α α

α

n const h

n n n

Size of the biggest hub is of order O(N). Most nodes can be connected within two steps, thus the average path length will be independent of the network size. The average path length increases slower than

• logarithmically. In Gnp all nodes have comparable degree,

thus most paths will have comparable length. In a scale- free network vast majority of the path go through the few high degree hubs, reducing the distances between nodes. Some models produce 𝛽 = 3. This was first derived by Bollobas et al. for the network diameter in the context of a dynamical model, but it holds for the average path length as well.

The second moment of the distribution is finite, thus in

many ways the network behaves as a random network. Hence the average path length follows the result that we derived for the random network model earlier.

Degree exponent

• Avg. path

length Ultra small world Small world

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 12

𝛽 = 1 Second moment 𝑙2 diverges 𝑙2 finite Average 𝑙 diverges 𝑙 finite Ultra small world behavior Small world Behaves like a random network The scale-free behavior is relevant Regime full of anomalies…

web web internet actor collaboration metabolic citation

𝛽 = 2 𝛽 = 3

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 13

 How does network

connectivity change as nodes get removed?

[Albert et al. 00; Palmer et al. 01]

 Nodes can be removed:

• Random failure:
• Remove nodes uniformly at random
• Targeted attack:
• Remove nodes in order of decreasing degree

 This is important for robustness of the internet

as well as epidemiology

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 17

 Real networks are resilient to random failures  Gnp has better resilience to targeted attacks

• Need to remove all pages of degree >5 to disconnect the Web
• But this is a very small fraction of all web pages

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 18

Fraction of removed nodes Mean path length Fraction of removed nodes Random failures Targeted attack Gnp network AS network Random failures Targeted attack

 There is no universal degree exponent

characterizing all networks

 We need growth and the preferential attachment

for the emergence of scale-free property

• The mechanism is domain dependent
• Many processes give rise to scale-free networks

 Modeling real networks:

• Identify microscopic processes that occur in the network
• Measure their frequency from real data
• Develop dynamical models that capture these processes
• If the model is correct, it should predict the observations

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 19

 Copying mechanism (directed network)

• Select a node and an edge of this node
• Attach to the endpoint of this edge

 Walking on a network (directed network)

• The new node connects to a node, then to every
• first, second, … neighbor of this node

 Attaching to edges

• Select an edge and attach to both endpoints of this edge

 Node duplication

• Duplicate a node with all its edges
• Randomly prune edges of new node

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 20

 Preferential attachment is a model

• f a growing network

 Can we find a more realistic model?  What governs network growth & evolution?

• P1) Node arrival process:
• When nodes enter the network
• P2) Edge initiation process:
• Each node decides when to initiate an edge
• P3) Edge destination process:
• The node determines destination of the edge

[Leskovec, Backstrom, Kumar, Tomkins, 2008]

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 22

 4 online social networks with

exact edge arrival sequence

• For every edge (u,v) we know exact

time of the creation tuv

 Directly observe mechanisms leading

to global network properties

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 23

(F) (D) (A) (L)

and so on for millions…

[Leskovec et al., KDD ’08]

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 24

(F) (D) (A) (L)

Flickr: Exponential Delicious: Linear Answers: Sub-linear LinkedIn: Quadratic

 How long do nodes live?

• Node life-time is the time between the 1st

and the last edge of a node

 How do nodes “wake up” to create links?

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 25

time 1st edge

• f node i

Last edge

• f node i

Lifetime of a node time 1st edge

• f node i

Last edge

• f node i

Node i creates edges

 Lifetime a:

Time between node’s first and last edge

LinkedIn

Node lifetime is exponentially distributed: 𝑞𝑚 𝑏 = 𝜇𝑓−𝜇𝜇

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 26

 How do nodes “wake up” to create edges?

• Edge gap 𝜺𝒆 𝒗 : time between 𝑒th and 𝑒 + 1st

edge of node 𝑗:

• Let 𝑢𝑒 𝑗 be the creation time of 𝑒-th edge of node 𝑗
• 𝜀𝑒 𝑣 = 𝑢𝑒+1 𝑣 − 𝑢𝑒 𝑣
• 𝜺𝒆 is a distribution (histogram) of 𝜺𝒆 𝒗 over all nodes 𝑣

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 27

time 1st edge

• f node i

Last edge

• f node i

𝜀1 𝑗 𝜀2 𝑗 𝜀3 𝑗 Node u 𝜀1 𝑣 Node v 𝜀1 𝑤 Node w 𝜀1 𝑥

β α

δ δ

− −

∝ e pg

1 1)

(

Edge gap 𝜺𝒆: inter-arrival time between 𝑒th and 𝑒 + 1st edge

LinkedIn

For every d we make a separate histogram

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 28

Edge gap, 𝜀1 Edge gap probability P(𝜀1)

 How do 𝜷 and 𝜸 change as a function of 𝒆?

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 29

d de

p

d d g β α

δ δ

− −

∝ ) (

To each plot of 𝜺𝒆 fit:

𝜷 is constant! 𝜸 linearly increases!

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 30

 𝜷 const., 𝜸 linear in 𝒆. What does this mean?  Gaps get smaller with 𝒆!

Degree 𝑒 = 1 𝑒 = 3 𝑒 = 2 Log 𝜺𝒆 Log 𝒒𝒉(𝜺𝒆)

d d d g

e p

⋅ − −

∝

β α

δ δ ) (

α

δ

−

∝

d

 Source node i wakes up and creates an edge  How does i select a target node j?

• What is the degree of the target j?
• Does preferential attachment really hold?
• How many hops away is the target j?
• Are edges attaching locally?

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 31

2 3 4

 Are edges more likely to connect to higher

degree nodes? YES!

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 32

τ

k k pe ∝ ) (

Gnp PA Flickr

Network τ Gnp PA 1 Flickr 1 Delicious 1 Answers 0.9 LinkedIn 0.6

[Leskovec et al., KDD ’08]

u w v

 Just before the edge (u,w) is placed how

many hops are between u and w?

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 33

Network % Δ Flickr 66% Delicious 28% Answers 23% LinkedIn 50% Fraction of triad closing edges

Real edges are local! Most of them close triangles!

Gnp PA Flickr

[Leskovec et al., KDD ’08]

 Focus only on triad-closing edges  New triad-closing edge (u,w) appears next  Model this as 2 independent choices:

• 1. u choses neighbor v
• 2. v choses neighbor w

and connect u to w

• E.g.: Under Random-Random:
• 𝑞 𝑣, 𝑥 =

1 5 ⋅ 1 2 + 1 5 ⋅ 1 = 3 10

 Under a particular pair of “strategies”:

Likelihood of the graph = ∏ 𝑞 𝑣, 𝑥

𝑣,𝑥 ∈𝐹

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 34

u w v v’

[Leskovec et al., KDD ’08]

 Improvement in log-likelihood over baseline:

• Baseline: Pick a random node 2 hops away

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 35

Strategy to select v (1st node) Select w (2nd node)

Strategies to pick a neighbor:

• random: uniformly at random
• deg: proportional to its degree
• com: prop. to the number of common friends
• last: prop. to time since last activity
• comlast: prop. to com*last

u w v

[Leskovec et al., KDD ’08]

 The model of network evolution

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu

Process Model P1) Node arrival

• Node arrival function is given

P2) Edge initiation

• Node lifetime is exponential
• Edge gaps get smaller as the

degree increases P3) Edge destination Pick edge destination using random-random

10/31/2012 36

[Leskovec et al., KDD ’08]

 Theorem: Exponential node lifetimes and

power-law with exponential cutoff edge gaps lead to power-law degree distributions

 Comments:

• The proof is based on a combination of

exponentials (see HW3)

• Interesting as temporal behavior predicts a

structural network property

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 37

[Leskovec et al., KDD ’08]

 Given the model one can take an existing

network continue its evolution

 Compare true and predicted (based on the

theorem) degree exponent:

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 38

 How do networks evolve at the macro level?

• What are global phenomena of network growth?

 Questions:

• What is the relation between the number of nodes

n(t) and number of edges e(t) over time t?

• How does diameter change as the network grows?
• How does degree distribution evolve as the

network grows?

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 39

 N(t) … nodes at time t  E(t) … edges at time t  Suppose that

N(t+1) = 2 * N(t)

 Q: what is

E(t+1) =2 * E(t)

 A: over-doubled!

• But obeying the Densification Power Law

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu

[Leskovec et al., KDD 05]

40

 Networks are denser over time  Densification Power Law:

a … densification exponent (1 ≤ a ≤ 2)

 What is the relation between

the number of nodes and the edges over time?

 First guess: constant average

degree over time

Internet Citations a=1.2 a=1.6

N(t) E(t) N(t) E(t)

41 10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu

[Leskovec et al., KDD 05]

 Densification Power Law

• the number of edges grows faster than the

number of nodes – average degree is increasing

a … densification exponent: 1 ≤ a ≤ 2:

• a=1: linear growth – constant out-degree

(traditionally assumed)

• a=2: quadratic growth – clique

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 42

• r

equivalently

[Leskovec et al. KDD 05]

 Prior models and intuition say

that the network diameter slowly grows (like log N, log log N)

time diameter diameter size of the graph Internet Citations

 Diameter shrinks over time

• as the network grows the

distances between the nodes slowly decrease

43 10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu

[Leskovec et al. KDD 05]

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu

diameter size of the graph

Erdos-Renyi random graph

Densification exponent a =1.3

Densifying random graph has increasing diameter⇒ There is more to shrinking diameter than just densification Is shrinking diameter just a consequence of densification?

[Leskovec et al. TKDD 07]

44

Is it the degree sequence? Compare diameter of a:

• True network (red)
• Random network with

the same degree distribution (blue)

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 45

size of the graph diameter Citations

Densification + degree sequence give shrinking diameter

 How does degree distribution evolve to allow

for densification?

 Option 1) Degree exponent 𝛽𝑢 is constant:

• Fact 1: For 1 < 𝛽𝑢 < 2 constant, then: 𝒃 = 𝟑/𝜷

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu

Email network [Leskovec et al. TKDD 07]

46

A consequence of what we learned in last class: ■ Power-laws with exponents <2 have infinite expectations. ■ So, by maintaining constant degree exponent 𝛽 the average degree grows.

 How does degree distribution evolve to allow

for densification?

 Option 2) 𝛽𝑢 evolves with graph size 𝑜:

• Fact 2: For 𝛽𝑢 =

4𝑜𝑢

𝑏−1−1

2𝑜𝑢

𝑏−1−1

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 47

Citation network [Leskovec et al. TKDD 07] Remember, expected degree is: 𝐹 𝑦 = 𝛽 − 1 𝛽 − 2 𝑦𝑛 So 𝛽 has to decay as as function of graph size for the avg. degree to go up

 Want to model graphs that density and have

shrinking diameters

 Intuition:

• How do we meet friends at a party?
• How do we identify references when writing

papers?

10/31/2012 48

[Leskovec et al. TKDD 07]

v w

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu

 The Forest Fire model has 2 parameters:

• p … forward burning probability
• r … backward burning probability

 The model:

• Each turn a new node v arrives
• Uniformly at random chooses an “ambassador” w
• Flip 2 geometric coins to determine the number of

in- and out-links of w to follow

• “Fire” spreads recursively until it dies
• New node v links to all burned nodes

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 49 10/31/2012

[Leskovec et al. TKDD 07] Geometric distribution:

 Forest Fire generates graphs that densify

and have shrinking diameter

10/31/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 50

densification diameter 1.32 N(t) E(t) N(t) diameter

 Forest Fire also generates graphs with

power-law degree distribution

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 51

in-degree

• ut-degree

log count vs. log in-degree log count vs. log out-degree

10/31/2012

 Fix backward

probability r and vary forward burning prob. p

 Notice a sharp

transition between sparse and clique-like graphs

 Sweet spot is

very narrow

Sparse graph Clique-like graph Increasing diameter Decreasing diameter Constant diameter

Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 52 10/31/2012