Algorithms for Big Data (XIII) Chihao Zhang Shanghai Jiao Tong - - PowerPoint PPT Presentation

Algorithms for Big Data (XIII)

Chihao Zhang

Shanghai Jiao Tong University

• Dec. 13, 2019

Algorithms for Big Data (XIII) 1/11

Review

We studied random walks on general graphs using spectral decomposition. We introduced the notion of electrical networks. We derived bounds on the cover time of random walks.

Algorithms for Big Data (XIII) 2/11

Review

We studied random walks on general graphs using spectral decomposition. We introduced the notion of electrical networks. We derived bounds on the cover time of random walks.

Algorithms for Big Data (XIII) 2/11

Review

We studied random walks on general graphs using spectral decomposition. We introduced the notion of electrical networks. We derived bounds on the cover time of random walks.

Algorithms for Big Data (XIII) 2/11

Review

We studied random walks on general graphs using spectral decomposition. We introduced the notion of electrical networks. We derived bounds on the cover time of random walks.

Algorithms for Big Data (XIII) 2/11

Electrical Network

Now we formally justify the electrical network argument used last week. For an edge with weight , we define its resistance . For an edge , we can assign numbers i i as the current on the edge. The collection of currents is required to satisfy Kirchhofg’s law. Ohm’s law is used to define the potential drop between two ends of an edge.

Algorithms for Big Data (XIII) 3/11

Electrical Network

Now we formally justify the electrical network argument used last week. For an edge with weight , we define its resistance . For an edge , we can assign numbers i i as the current on the edge. The collection of currents is required to satisfy Kirchhofg’s law. Ohm’s law is used to define the potential drop between two ends of an edge.

Algorithms for Big Data (XIII) 3/11

Electrical Network

Now we formally justify the electrical network argument used last week. For an edge with weight we, we define its resistance re = w−1

e .

For an edge , we can assign numbers i i as the current on the edge. The collection of currents is required to satisfy Kirchhofg’s law. Ohm’s law is used to define the potential drop between two ends of an edge.

Algorithms for Big Data (XIII) 3/11

Electrical Network

Now we formally justify the electrical network argument used last week. For an edge with weight we, we define its resistance re = w−1

e .

For an edge {u, v}, we can assign numbers i(u, v) = −i(v, u) as the current on the edge. The collection of currents is required to satisfy Kirchhofg’s law. Ohm’s law is used to define the potential drop between two ends of an edge.

Algorithms for Big Data (XIII) 3/11

Electrical Network

Now we formally justify the electrical network argument used last week. For an edge with weight we, we define its resistance re = w−1

e .

For an edge {u, v}, we can assign numbers i(u, v) = −i(v, u) as the current on the edge. The collection of currents is required to satisfy Kirchhofg’s law. Ohm’s law is used to define the potential drop between two ends of an edge.

Algorithms for Big Data (XIII) 3/11

Electrical Network

Now we formally justify the electrical network argument used last week. For an edge with weight we, we define its resistance re = w−1

e .

For an edge {u, v}, we can assign numbers i(u, v) = −i(v, u) as the current on the edge. The collection of currents is required to satisfy Kirchhofg’s law. Ohm’s law is used to define the potential drop between two ends of an edge.

Algorithms for Big Data (XIII) 3/11

Matrix Form

It is instructive to express physical laws in the matrix form. We use an ordered pair satisfying to represent an edge . The signed edge-vertex adjacency matrix is defined as if if

• therwise.

Let be diag .

Algorithms for Big Data (XIII) 4/11

Matrix Form

It is instructive to express physical laws in the matrix form. We use an ordered pair satisfying to represent an edge . The signed edge-vertex adjacency matrix is defined as if if

• therwise.

Let be diag .

Algorithms for Big Data (XIII) 4/11

Matrix Form

It is instructive to express physical laws in the matrix form. We use an ordered pair (u, v) satisfying u ≤ v to represent an edge {u, v} ∈ E. The signed edge-vertex adjacency matrix is defined as if if

• therwise.

Let be diag .

Algorithms for Big Data (XIII) 4/11

Matrix Form

It is instructive to express physical laws in the matrix form. We use an ordered pair (u, v) satisfying u ≤ v to represent an edge {u, v} ∈ E. The signed edge-vertex adjacency matrix U ∈ {0, 1, −1}E×V is defined as U((u, v), w) =      1 if w = u −1 if w = v

• therwise.

Let be diag .

Algorithms for Big Data (XIII) 4/11

Matrix Form

It is instructive to express physical laws in the matrix form. We use an ordered pair (u, v) satisfying u ≤ v to represent an edge {u, v} ∈ E. The signed edge-vertex adjacency matrix U ∈ {0, 1, −1}E×V is defined as U((u, v), w) =      1 if w = u −1 if w = v

• therwise.

Let W ∈ RE×E be diag(w(e1), . . . , w(e|E|)).

Algorithms for Big Data (XIII) 4/11

We use i ∈ RE to denote the vector of currents, v ∈ RV to denote the vector of voltages. It holds that i v We use iext to denote the amount of current entering externally. Then iext i , and iext

Ti T

v If iext , we call it a internal node, otherwise, we call it a boundary node.

Algorithms for Big Data (XIII) 5/11

We use i ∈ RE to denote the vector of currents, v ∈ RV to denote the vector of voltages. It holds that i = W · U · v. We use iext to denote the amount of current entering externally. Then iext i , and iext

Ti T

v If iext , we call it a internal node, otherwise, we call it a boundary node.

Algorithms for Big Data (XIII) 5/11

We use i ∈ RE to denote the vector of currents, v ∈ RV to denote the vector of voltages. It holds that i = W · U · v. We use iext(u) to denote the amount of current entering u externally. Then iext i , and iext

Ti T

v If iext , we call it a internal node, otherwise, we call it a boundary node.

Algorithms for Big Data (XIII) 5/11

We use i ∈ RE to denote the vector of currents, v ∈ RV to denote the vector of voltages. It holds that i = W · U · v. We use iext(u) to denote the amount of current entering u externally. Then iext(u) = ∑

v∈N(u) i(u, v), and

iext

Ti T

v If iext , we call it a internal node, otherwise, we call it a boundary node.

Algorithms for Big Data (XIII) 5/11

We use i ∈ RE to denote the vector of currents, v ∈ RV to denote the vector of voltages. It holds that i = W · U · v. We use iext(u) to denote the amount of current entering u externally. Then iext(u) = ∑

v∈N(u) i(u, v), and

iext = UTi = UT · W · U · v. If iext , we call it a internal node, otherwise, we call it a boundary node.

Algorithms for Big Data (XIII) 5/11

We use i ∈ RE to denote the vector of currents, v ∈ RV to denote the vector of voltages. It holds that i = W · U · v. We use iext(u) to denote the amount of current entering u externally. Then iext(u) = ∑

v∈N(u) i(u, v), and

iext = UTi = UT · W · U · v. If iext(u) = 0, we call it a internal node, otherwise, we call it a boundary node.

Algorithms for Big Data (XIII) 5/11

Graph Laplacian

The matrix

T

is again graph Laplacian. Consider the spectral decomposition of : v vT Using the decomposition, the equation becomes to v v vT v where iext v and v v .

Algorithms for Big Data (XIII) 6/11

Graph Laplacian

The matrix L ≜ UTWU is again graph Laplacian. Consider the spectral decomposition of : v vT Using the decomposition, the equation becomes to v v vT v where iext v and v v .

Algorithms for Big Data (XIII) 6/11

Graph Laplacian

The matrix L ≜ UTWU is again graph Laplacian. Consider the spectral decomposition of L: L = ∑

i>1

λivivT

i .

Using the decomposition, the equation becomes to v v vT v where iext v and v v .

Algorithms for Big Data (XIII) 6/11

Graph Laplacian

The matrix L ≜ UTWU is again graph Laplacian. Consider the spectral decomposition of L: L = ∑

i>1

λivivT

i .

Using the decomposition, the equation becomes to ∑

i≥1

aivi = (∑

i>1

λivivT

i

)  ∑

i≥1

bivi   , where iext = ∑

i≥1 aivi and v = ∑ i≥1 bivi.

Algorithms for Big Data (XIII) 6/11

Therefore, we must have a1 = 0, which means the current entering the network is equal to the current leaving the network! Define the Moore-Penrose pseudo-inverse of v vT Given iext, we can compute v as long as we can compute . We shifu v so that v iext

Algorithms for Big Data (XIII) 7/11

Therefore, we must have a1 = 0, which means the current entering the network is equal to the current leaving the network! Define the Moore-Penrose pseudo-inverse of L L+ = ∑

i>1

λ−1

i vivT i .

Given iext, we can compute v as long as we can compute . We shifu v so that v iext

Algorithms for Big Data (XIII) 7/11

Therefore, we must have a1 = 0, which means the current entering the network is equal to the current leaving the network! Define the Moore-Penrose pseudo-inverse of L L+ = ∑

i>1

λ−1

i vivT i .

Given iext, we can compute v as long as we can compute L+. We shifu v so that v iext

Algorithms for Big Data (XIII) 7/11

Therefore, we must have a1 = 0, which means the current entering the network is equal to the current leaving the network! Define the Moore-Penrose pseudo-inverse of L L+ = ∑

i>1

λ−1

i vivT i .

Given iext, we can compute v as long as we can compute L+. We shifu v so that v = L+iext.

Algorithms for Big Data (XIII) 7/11

Effective Resistance

We are now able to formally define efgective resistance.

eff

e e

T

e e To see this, assuming one unit of current enters and leaves : v e e On the other hand, v v e e

Tv

e e

T

e e

Algorithms for Big Data (XIII) 8/11

Effective Resistance

We are now able to formally define efgective resistance.

eff

e e

T

e e To see this, assuming one unit of current enters and leaves : v e e On the other hand, v v e e

Tv

e e

T

e e

Algorithms for Big Data (XIII) 8/11

Effective Resistance

We are now able to formally define efgective resistance. Reff(u, v) ≜ (eu − ev)TL+(eu − ev). To see this, assuming one unit of current enters and leaves : v e e On the other hand, v v e e

Tv

e e

T

e e

Algorithms for Big Data (XIII) 8/11

Effective Resistance

We are now able to formally define efgective resistance. Reff(u, v) ≜ (eu − ev)TL+(eu − ev). To see this, assuming one unit of current enters u and leaves v: v = L+(eu − ev). On the other hand, v v e e

Tv

e e

T

e e

Algorithms for Big Data (XIII) 8/11

Effective Resistance

We are now able to formally define efgective resistance. Reff(u, v) ≜ (eu − ev)TL+(eu − ev). To see this, assuming one unit of current enters u and leaves v: v = L+(eu − ev). On the other hand, v(u) − v(v) = (eu − ev)Tv = (eu − ev)TL+(eu − ev).

Algorithms for Big Data (XIII) 8/11

Note that L+ is positive semi-definite, we can define L+/2 = ∑

i>1

λ−1/2vi. Then we can write v v e e

Tv

e e

T

e e e e Examples: Series and Parallel graphs.

Algorithms for Big Data (XIII) 9/11

Note that L+ is positive semi-definite, we can define L+/2 = ∑

i>1

λ−1/2vi. Then we can write v(u) − v(v) = (eu − ev)Tv = (eu − ev)TL+(eu − ev) = ∥L+/2(eu − ev)∥2

2.

Examples: Series and Parallel graphs.

Algorithms for Big Data (XIII) 9/11

Note that L+ is positive semi-definite, we can define L+/2 = ∑

i>1

λ−1/2vi. Then we can write v(u) − v(v) = (eu − ev)Tv = (eu − ev)TL+(eu − ev) = ∥L+/2(eu − ev)∥2

2.

Examples: Series and Parallel graphs.

Algorithms for Big Data (XIII) 9/11

Approximating Effective Resistance

Directly computing efgective resistance requires to compute , which is costly. We can view e and e as two vectors in and approximate their distance using metric embedding technique. Recall in Lecture 6, we learnt:

Theorem

For any and any positive integer , consider a set of points . There exists an matrix where log satisfying x y x y x y x y

Algorithms for Big Data (XIII) 10/11

Approximating Effective Resistance

Directly computing efgective resistance requires to compute L+, which is costly. We can view e and e as two vectors in and approximate their distance using metric embedding technique. Recall in Lecture 6, we learnt:

Theorem

For any and any positive integer , consider a set of points . There exists an matrix where log satisfying x y x y x y x y

Algorithms for Big Data (XIII) 10/11

Approximating Effective Resistance

Directly computing efgective resistance requires to compute L+, which is costly. We can view L+/2eu and L+/2ev as two vectors in Rn and approximate their distance using metric embedding technique. Recall in Lecture 6, we learnt:

Theorem

For any and any positive integer , consider a set of points . There exists an matrix where log satisfying x y x y x y x y

Algorithms for Big Data (XIII) 10/11

Approximating Effective Resistance

Directly computing efgective resistance requires to compute L+, which is costly. We can view L+/2eu and L+/2ev as two vectors in Rn and approximate their distance using metric embedding technique. Recall in Lecture 6, we learnt:

Theorem

For any 0 < ε < 1

2 and any positive integer m, consider a set of m points S ⊆ Rn. There

exists an matrix A ∈ Rk×n where k = O ( ε−2 log m ) satisfying ∀x, y ∈ S, (1 − ε)∥x − y∥ ≤ ∥Ax − Ay∥ ≤ (1 + ε)∥x − y∥.

Algorithms for Big Data (XIII) 10/11

In our proof of JLT, each entry of the matrix A is from N(0, 1/k). We only need to show how to compute efgiciently… Let , then

T

. Therefore e e

eff

. We only need to solve

• linear equations in

to obtain .

Algorithms for Big Data (XIII) 11/11

In our proof of JLT, each entry of the matrix A is from N(0, 1/k). We only need to show how to compute AL+/2 efgiciently… Let , then

T

. Therefore e e

eff

. We only need to solve

• linear equations in

to obtain .

Algorithms for Big Data (XIII) 11/11

In our proof of JLT, each entry of the matrix A is from N(0, 1/k). We only need to show how to compute AL+/2 efgiciently… Let L′ ≜ W1/2U, then (L′)TL′ = L+. Therefore e e

eff

. We only need to solve

• linear equations in

to obtain .

Algorithms for Big Data (XIII) 11/11

In our proof of JLT, each entry of the matrix A is from N(0, 1/k). We only need to show how to compute AL+/2 efgiciently… Let L′ ≜ W1/2U, then (L′)TL′ = L+. Therefore ∥L′(eu − ev)∥2

2 = Reff(u, v).

We only need to solve

• linear equations in

to obtain .

Algorithms for Big Data (XIII) 11/11

In our proof of JLT, each entry of the matrix A is from N(0, 1/k). We only need to show how to compute AL+/2 efgiciently… Let L′ ≜ W1/2U, then (L′)TL′ = L+. Therefore ∥L′(eu − ev)∥2

2 = Reff(u, v).

We only need to solve d-linear equations in L to obtain AL′L.

Algorithms for Big Data (XIII) 11/11