Methods for estimating the diagonal of matrix functions Jesse - - PowerPoint PPT Presentation

Methods for estimating the diagonal of matrix functions

Methods for estimating the diagonal of matrix functions

Jesse Laeuchli Andreas Stathopoulos CSC 2016

1 / 24

Methods for estimating the diagonal of matrix functions

The Problem

Given a large matrix A, and a function f , find diag(f(A)) ,or trace(f(A)) = n

i=1 f(A)ii = n i=1 f(λi)

Some important examples of f

– f (A) = A−1 – f (A) = Ak – f (A) = exp(A) – f (A) = log(A)

2 / 24

Methods for estimating the diagonal of matrix functions

Applications

Compute Diagonals of Ak/exp(A) Count Triangles/Polygons–Higher distance paths, higher powers of A Network Centrality

3 / 24

Methods for estimating the diagonal of matrix functions

Applications

Compute Diagonals of A−1

−20 −10 10 20 30 40 50 60 2 4 6 8 10 12 14 16

y = − 0.0003*x2 + 0.16*x + 5

Observation LSQ

Statistics

4 / 24

Methods for estimating the diagonal of matrix functions

Applications

Compute Diagonals of A−1 Uncertainty Quantification

5 / 24

Methods for estimating the diagonal of matrix functions

Exact Methods

What if we compute it exactly? LU Decomposition Eigendecomposition Recursive Factorizations Takahashi’s Equations All too slow for large matrices

6 / 24

Methods for estimating the diagonal of matrix functions

Statistical Trace Estimator

Exact methods infeasible, so we resort to statistical approaches trace(f(A)) = E[zTf(A)z] ≈ i=m

i=0 zT i f(A)zi

m diag(f(A)) = E[z ⊙ f(A)z] ≈ (

m

• i=1

zi ⊙ f(A)zi) ⊘ (

m

• i=1

zi ⊙ zi) a b

• ⊙

c d

• =

a ∗ c b ∗ d

• ,

a b

• ⊘

c d

• =

a/c b/d

• 7 / 24

Methods for estimating the diagonal of matrix functions

How to pick z? Random

Random Methods

– Gaussian{z = from Gaussian Distribution } – Hutchinson{z = 1, −1 probability 1/2 } – Canoncial basis ei with random i – Mixing of diagonals DFTnei,Hadamardnei with random i

Estimator Variance of the Sample Gaussian 2A2

F

Hutchinson’s 2(A2

F − n i=1 A2 ii)

Unit Vector n n

i=1 A2 ii − trace2(A)

8 / 24

Methods for estimating the diagonal of matrix functions

How to pick z? Deterministic

Deterministic

– Hadamardnei with i = 1 : n

20 40 60 80 100 120 20 40 60 80 100 120 nz = 1024 20 40 60 80 100 120 20 40 60 80 100 120 nz = 512

H1 =

• 1
• H2 =

1 1 1 −1

• H2k =

H2k−1 H2k−1 H2k−1 −H2k−1

• = H2 ⊗ H2k−1

9 / 24

Methods for estimating the diagonal of matrix functions

Probing

Color the associated graph of A, reorder nodes with the same color to be adjacent Can then recover the diagonal of an m-colorable matrix with exactly m vectors xm

i

=

• 1,

if i ∈ Cm 0,

• therwise.

If we color graph of f (A), then we can recover diag(f(A))

• 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1

• 10 / 24

Methods for estimating the diagonal of matrix functions

Probing f (A)

Computing f (A) is hard. If we had it, we would be done f (A) is dense. It takes too many colors to probe Look at structure of polynomial pk(A) approximating f (A)

11 / 24

Methods for estimating the diagonal of matrix functions

Problems With Probing-What We Address

Even lower order Ak expensive to compute and store

1 2 3 4 5 6 7 8 2 4 6 8 10x 10

6

Powers of L NNZ Powers of L

12 / 24

Methods for estimating the diagonal of matrix functions

Problems With Probing-What We Address

Probing vectors for Ak not guaranteed to be a subset of vectors for Ak+1

13 / 24

Methods for estimating the diagonal of matrix functions

Hierarchical Coloring

What is Hierarchical Coloring?

– Colors must be nested – Colors split into the same number of new colors – Ensures reusability of probing vectors

14 / 24

Methods for estimating the diagonal of matrix functions

Hierarchical Coloring via Multilevel

How to color a arbitrary graph? Use multi-level strategy. Problems if merging is done naively.

1 2 3 4 5 6 7 0 1 2 3 1 2 3 4 5 6 7 4 5 6 7 1 2 3

Same color neighbor at distance−8

Two green nodes still at distance−2 after 3 levels

15 / 24

Methods for estimating the diagonal of matrix functions

Hierarchical Coloring via Multilevel

Our strategy, merge distance 1 nodes, and distance 2 neighbourhoods N−1 N−2 N−1 N−2

16 / 24

Methods for estimating the diagonal of matrix functions

Hierarchical Coloring via Multilevel

Example Graph. One level of merging

17 / 24

Methods for estimating the diagonal of matrix functions

Ensure Hierarchical Coloring

Create Mixed-Radix Coordinate Interpret as color at each level Ensures colorings are hierarchical

18 / 24

Methods for estimating the diagonal of matrix functions

Statistical Considerations

How well should we expect to do with HP? Pure statistical is O( 1

√s )

Hierarchical Probing depends on structure, g(x) describes fall-off of elements g(x) = c

– O( 1

√s )

g(x) = 1 − x

– O( 1

s )

If there is no structure Hierarchical Probing does as well as the statistical methods

19 / 24

Methods for estimating the diagonal of matrix functions

Different Coloring Methods

Top eigenvectors can be used to divide graph into bipartite groups in multiple ways

Eigenvector

Coloring From Probing

20 / 24

Methods for estimating the diagonal of matrix functions

Covariance Matrix

10 10

1

10

2

10

3

10

4

10

−20

10

−15

10

−10

10

−5

10 10

5

Colors Variance Merge/Greedy Color Statistical Probing

21 / 24

Methods for estimating the diagonal of matrix functions

Uncertainty Quantification

10 10

1

10

2

10

3

10

4

10

−4

10

−2

10 10

2

Colors Variance Merge/Greedy Color Statistical Probing

22 / 24

Methods for estimating the diagonal of matrix functions

Wiki-Vote

10 10

1

10

2

10

3

10

4

10

−5

10

−4

10

−3

10

−2

10

−1

10 Colors Variance Merge/Greedy Color Merge/Spectral Color Statistical Probing

23 / 24

Methods for estimating the diagonal of matrix functions

Conclusion

Exploits structure of matrix Statistical bounds More efficient than classical probing

24 / 24