An Empirical Study of Cycle Toggling Based Laplacian Solvers Kevin - - PowerPoint PPT Presentation

▶
an empirical study of cycle toggling based laplacian
SMART_READER_LITE
LIVE PREVIEW

An Empirical Study of Cycle Toggling Based Laplacian Solvers Kevin - - PowerPoint PPT Presentation

Jun 07, 2023 146 likes •677 views

An Empirical Study of Cycle Toggling Based Laplacian Solvers Kevin Deweese 1 John Gilbert 1 Gary Miller 2 Richard Peng 3 Hao Ran Xu 4 Shen Chen Xu 2 1 UCSB 2 Carnegie Mellon 3 Georgia Tech 4 MIT SIAM Workshop on Combinatorial Scientific Computing,

slide-1
SLIDE 1

An Empirical Study of Cycle Toggling Based Laplacian Solvers

Kevin Deweese1 John Gilbert1 Gary Miller2 Richard Peng3 Hao Ran Xu4 Shen Chen Xu2

1UCSB 2Carnegie Mellon 3Georgia Tech 4MIT

SIAM Workshop on Combinatorial Scientific Computing, 2016

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 1 / 30

slide-2
SLIDE 2

Problem Description

Our focus: Solve the the system of equations Lx = b where L is a graph Laplacian matrix

3 4 1 2

2 1 1 1 1     3 −2 −1 −2 4 −1 −1 −1 −1 3 −1 −1 −1 2    

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 2 / 30

slide-3
SLIDE 3

Electrical Networks

Edge weights of Laplacians are actually conductances, or inverse resistances we = 1/re Right hand side b contains current demands at every vertex Left hand side x contains potential, or voltage at every vertex Can also consider a flow, or current

  • n every edge

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 3 / 30

slide-4
SLIDE 4

Applications

Graphs with regular degree structure, 2D/3D meshes

Finite element analysis

Electrical and thermal conductivity Fluid flow modeling

Image processing

Image segmentation, inpainting, regression, classification

Graphs with irregular degree, problems in network analysis

Maximum flow problems Graph sparsification Spectral clustering

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 4 / 30

slide-5
SLIDE 5

Solvers With Good Asymptotic Complexity

Primal (solves for vertex potentials)

Linear times polylog. Spielman and Teng, 2006 Nearly m log n. Koutis, Miller, and Peng, 2011 Nearly m log1/2 n. Cohen et al., 2016

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 5 / 30

slide-6
SLIDE 6

Solvers With Good Asymptotic Complexity

Primal (solves for vertex potentials)

Linear times polylog. Spielman and Teng, 2006 Nearly m log n. Koutis, Miller, and Peng, 2011 Nearly m log1/2 n. Cohen et al., 2016

Dual (solves for edge flows)

A simple, nearly m log2 n, combinatorial algorithm. Kelner, Orecchia, Sidford, and Zhu, 2013

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 5 / 30

slide-7
SLIDE 7

Solvers With Good Asymptotic Complexity

Primal (solves for vertex potentials)

Linear times polylog. Spielman and Teng, 2006 Nearly m log n. Koutis, Miller, and Peng, 2011 Nearly m log1/2 n. Cohen et al., 2016

Dual (solves for edge flows)

A simple, nearly m log2 n, combinatorial algorithm. Kelner, Orecchia, Sidford, and Zhu, 2013 Simplest

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 5 / 30

slide-8
SLIDE 8

Our Goal

Provide some understanding of Kelner et al. (cycle toggling) implementations Introduce a useful class of test problems, heavy path graphs, for exploring these methods Examine performance behavior of different cycle toggling methods

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 6 / 30

slide-9
SLIDE 9

Kelner et al.’s Method (Cycle Toggling)

Select cycle (with probability proportional to stretch) from a fundamental cycle basis Update flows around cycle

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 7 / 30

slide-10
SLIDE 10

Kelner et al.’s Method (Cycle Toggling)

Select cycle (with probability proportional to stretch) from a fundamental cycle basis Update flows around cycle

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 7 / 30

slide-11
SLIDE 11

Kelner et al.’s Method (Cycle Toggling)

Select cycle (with probability proportional to stretch) from a fundamental cycle basis Update flows around cycle

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 7 / 30

slide-12
SLIDE 12

Kelner et al.’s Method (Cycle Toggling)

Select cycle (with probability proportional to stretch) from a fundamental cycle basis Update flows around cycle

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 7 / 30

slide-13
SLIDE 13

Kelner et al.’s Method (Cycle Toggling)

Select cycle (with probability proportional to stretch) from a fundamental cycle basis Update flows around cycle

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 7 / 30

slide-14
SLIDE 14

Kelner et al.’s Method (Cycle Toggling)

Select cycle (with probability proportional to stretch) from a fundamental cycle basis Update flows around cycle

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 7 / 30

slide-15
SLIDE 15

Cycle Toggle Methods

total cost = number of cycle toggles × cost per cycle toggle

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 8 / 30

slide-16
SLIDE 16

Cycle Toggle Methods

total cost = number of cycle toggles × cost per cycle toggle proportional to tree stretch

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 8 / 30

slide-17
SLIDE 17

Cycle Toggle Methods

total cost = number of cycle toggles × cost per cycle toggle clever implementations

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 8 / 30

slide-18
SLIDE 18

Heavy Path Graphs

Path graph + edges weighted such that the path is low-stretch tree Used to explore fundamental questions of cycle toggling approaches Can be tuned to have various stretch and spectral properties

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 9 / 30

slide-19
SLIDE 19

Cycle Toggle Methods

Support two operations

Query: (find voltage drop,

e refe along cycles)

Update: (alter flow of the cycle by ∆)

We consider two strategies

Single level with fancy data structures Multilevel divide-and-conquer

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 10 / 30

slide-20
SLIDE 20

Toggle Method 1: Single Level with Data Structures

Balanced binary search trees can be used to provide O(log n) query and update operations on intervals of a path graph. Create BST representing path intervals Store voltage drop, rf at every sub-interval Initialize a lazy tag at every interval to 0

1 2 3 4 5 6 7 0-7 0-4 4-7 0-2 2-4 0-1 1-2 2-3 3-4 4-6 6-7 4-5 5-6 Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 11 / 30

slide-21
SLIDE 21

Toggle Method 1: Single Level with Data Structures

Balanced binary search trees can be used to provide O(log n) query and update operations on intervals of a path graph. Query interval 1-4 Add sums of intervals 1-2 and 2-4

1 2 3 4 5 6 7 0-7 0-4 4-7 0-2 2-4 0-1 1-2 2-3 3-4 4-6 6-7 4-5 5-6 Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 11 / 30

slide-22
SLIDE 22

Toggle Method 1: Single Level with Data Structures

Balanced binary search trees can be used to provide O(log n) query and update operations on intervals of a path graph. Apply update ∆ to interval 1-4 by updating intervals 1-2, 2-4, and all ancestor intervals Set tag of intervals 1-4 and 1-2 to ∆

1 2 3 4 5 6 7 2-4 1-2 tag=Δ tag=Δ Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 11 / 30

slide-23
SLIDE 23

Toggle Method 1: Single Level with Data Structures

Balanced binary search trees can be used to provide O(log n) query and update operations on intervals of a path graph. Query interval 3-6 Encounter non-zero tag

1 2 3 4 5 6 7 0-7 0-4 4-7 0-2 2-4 0-1 1-2 2-3 3-4 4-6 6-7 4-5 5-6 tag=Δ Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 11 / 30

slide-24
SLIDE 24

Toggle Method 1: Single Level with Data Structures

Balanced binary search trees can be used to provide O(log n) query and update operations on intervals of a path graph. Push tag information to children Set tag to 0

1 2 3 4 5 6 7 0-7 0-4 4-7 0-2 2-4 0-1 1-2 2-3 3-4 4-6 6-7 4-5 5-6 tag=0 Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 11 / 30

slide-25
SLIDE 25

Toggle Method 1: Single level with Data Structures

Can be extended to general graphs BSTs can be combined with heavy light decomposition to yield O(log2 n) per update Can be improved to O(log n) with virtual trees

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 12 / 30

slide-26
SLIDE 26

Toggle Method 2: Multilevel Divide-and-Conquer

Preselect a batch of K cycles to update Use this knowledge to reduce problem size (contraction, path compression) Operate on the batch of cycles recursively Updating a batch is O(n log n) if K is O(n)

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 13 / 30

slide-27
SLIDE 27

Toggle Method 2: Multilevel Divide-and-Conquer

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 14 / 30

slide-28
SLIDE 28

Toggle Method 2: Multilevel Divide-and-Conquer

Sample cycles

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 14 / 30

slide-29
SLIDE 29

Toggle Method 2: Multilevel Divide-and-Conquer

Sample cycles Contract graph on selected cycles

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 14 / 30

slide-30
SLIDE 30

Toggle Method 2: Multilevel Divide-and-Conquer

Sample cycles Contract graph on selected cycles Remove degree-2 vertices

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 14 / 30

slide-31
SLIDE 31

Toggle Method 2: Multilevel Divide-and-Conquer

Sample cycles Contract graph on selected cycles Remove degree-2 vertices

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 14 / 30

slide-32
SLIDE 32

Experimental Setup: Methods to Compare

Single level, general graphs Single level, heavy path optimized Multilevel, general graphs Multilevel, heavy path optimized

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 15 / 30

slide-33
SLIDE 33

Experimental Setup: Methods to Compare

Single level, general graphs Single level, heavy path optimized Multilevel, general graphs Multilevel, heavy path optimized Want to compare average cycle update time

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 15 / 30

slide-34
SLIDE 34

Experimental Setup: Methods to Compare

Single level, general graphs Single level, heavy path optimized Multilevel, general graphs Multilevel, heavy path optimized Jacobi preconditioned conjugate gradient

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 15 / 30

slide-35
SLIDE 35

Experimental Setup: Heavy Path Graphs

Fixed Cycle Length (2 and 1000) Random Cycle Length 2D Mesh 3D Mesh

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 16 / 30

slide-36
SLIDE 36

Experimental Setup: Heavy Path Graphs

Edge weights chosen for different stretch behavior

Uniform stretch: Stretch is 1 for every cycle Exponential stretch: Stretch of every cycle is sampled from an exponential distribution

Graph size in vertices 5 × 104, 105, 5 × 105, 106

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 17 / 30

slide-37
SLIDE 37

Experimental Setup

Right hand sides

Random: Select x and form b = Lx (-1,1): Route one unit of flow from one endpoint of path to the other

Residual tolerance 10−5

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 18 / 30

slide-38
SLIDE 38

Performance Profile: Cycle Toggle Time

1 2 3 4 5 6 7 8 9 10

Relative Performance Distance

0.0 0.2 0.4 0.6 0.8 1.0

Fraction of Test Problems

Path-only Single Level General Single Level Path-only Multilevel General Multilevel

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 19 / 30

slide-39
SLIDE 39

Cycle Toggle Time

Path-only Path-only General General Single Level Multilevel Single Level Multilevel % of problems solver is best 100

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 20 / 30

slide-40
SLIDE 40

Cycle Toggle Time

Path-only Path-only General General Single Level Multilevel Single Level Multilevel % of problems solver is best 100 % within factor 2 of best 100 60 20

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 20 / 30

slide-41
SLIDE 41

Cycle Toggle Time

Path-only Path-only General General Single Level Multilevel Single Level Multilevel % of problems solver is best 100 % within factor 2 of best 100 60 20 % within factor 10 of best 100 100 100 80

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 20 / 30

slide-42
SLIDE 42

Performance Profile: Cycle Toggle Time

1 2 3 4 5 6 7 8 9 10

Relative Performance Distance

0.0 0.2 0.4 0.6 0.8 1.0

Fraction of Test Problems

Path-only Single Level General Single Level Path-only Multilevel General Multilevel

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 21 / 30

slide-43
SLIDE 43

PCG Comparison (to General Single Level)

101 102 103 104 105

PCG Time(s)

101 102 103 104 105

Toggle Time(s)

Uniform Stretch FixedLength-2 2D Mesh Random Exp Stretch FixedLength-1k 3D Mesh

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 22 / 30

slide-44
SLIDE 44

Weak Scaling

104 105 106

Path Length

10−7 10−6 10−5

Average Toggle Time(s)

Path-only Single level General Single level Path-only Multilevel General Multilevel

3D Mesh Exponential Stretch

104 105 106

Path Length

10−7 10−6 10−5

Average Toggle Time(s)

Path-only Single level General Single level Path-only Multilevel General Multilevel

Fixed Length 1k Uniform Stretch

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 23 / 30

slide-45
SLIDE 45

Timing Breakdown of Recursive Toggling

103 104 105 106 107

Path Length

10−8 10−7 10−6 10−5

Average Toggle Time(s)

Restrict Prolong Update Solve

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 24 / 30

slide-46
SLIDE 46

Summary of Results

Heavy path graphs are a useful model to consider

Simplify implementation details Cycle toggling outperforms PCG

Single level with fancy data structures performs better than multilevel recursive Check out our code and data at https://github.com/sxu/cycleToggling

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 25 / 30

slide-47
SLIDE 47

Future Work

Combine primal and dual solvers Examine floating point ops required for recursive cycle updates Further explore heavy path graphs

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 26 / 30

slide-48
SLIDE 48

Thank You

Henning Meyerhenke

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 27 / 30

slide-49
SLIDE 49

Heavy Light Decomposition

Arbitrarily root tree

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 28 / 30

slide-50
SLIDE 50

Heavy Light Decomposition

Arbitrarily root tree Mark child edges to largest subtree as heavy Maximal length paths of heavy edges are called heavy chains

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 28 / 30

slide-51
SLIDE 51

Heavy Light Decomposition

Arbitrarily root tree Mark child edges to largest subtree as heavy Maximal length paths of heavy edges are called heavy chains Mark other edges as light

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 28 / 30

slide-52
SLIDE 52

BST + HLD

A path from any vertex to the root intersects O(log n) heavy chains and O(log n) light edges O(1)-cost operations on light edges and O(log n)-cost operations

  • n heavy chains via binary search trees

Theoretical bound of O(log2 n) per operation, good running time experimentally

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 29 / 30

slide-53
SLIDE 53

Stretch Dependency

103 104 105 106 107 108

Total Stretch

103 104 105 106 107 108

Cycles log ǫ−1

Deweese et al. Cycle Toggling Laplacian Solvers CSC 2016 30 / 30