Faster Fully Dynamic Transitive Closure in Practice Kathrin Hanauer - PowerPoint PPT Presentation

Faster Fully Dynamic Transitive Closure in Practice Kathrin Hanauer Monika Henzinger Christian Schulz June 17, 2020 kathrin.hanauer@univie.ac.at

All-Pairs Reachability a.k.a. Transitive Closure Fully Dynamic Transitive Closure directed graph + sequence of operations: ? queries s � t edge insertions & deletions Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Query & Update Times (in O ) 2 Very Large Studies [FMNZ01, KZ08] m 1 Distinctly fastest on most instances: static graph traversal n 2 1 static graph traversal algorithms [DI08, Rod08, San04] √ n m √ n [RZ08] Strongest competitors: m 0 . 43 m 0 . 58 n [RZ08] two SCC-maintaining algorithms n 0 . 58 n 1 . 58 [San04] n 1 . 495 n 1 . 495 Few “real-world” graphs [San04] n m + n log n [RZ16] n 1 . 407 n 1 . 407 [vdBNS19] PRACTICE Our Idea THEORY Engineering algorithms that . . . use single-source reachability don’t maintain SCCs FULLY DYNAMIC profit from SCCs TRANSITIVE CLOSURE Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Supportive Vertices Observations Let v , s , t be vertices. R + ( v )/ R − ( v ): vertices reachable from/that can reach v ? Consider query s � t : s s t t R − ( v ) R − ( v ) R − ( v ) R − ( v ) R − ( v ) R − ( v ) v v v v v v t t R + ( v ) R + ( v ) R + ( v ) R + ( v ) R + ( v ) R + ( v ) s s t t s s (O1) (O2) (O3) ? v is a supportive vertex: R + ( v )/ R − ( v ) can help to answer s � t Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Supportive Vertices Algorithms General Outline ◮ Store list of supportive vertices L SV ∀ v ∈ L SV : maintain R + ( v ) and R − ( v ) via SSR; algorithms ◮ Updates (edge insertions & deletions): single-source/ single-sink forward to SSR algorithms reachability ◮ Query: ∀ v ∈ L SV : try to answer via (O1), (O2), (O3) fallback to static graph traversal s t R − ( v ) R − ( v ) R − ( v ) v v v t R + ( v ) R + ( v ) R + ( v ) s t s (O1) (O2) (O3) Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Supportive Vertices Algorithms SVA( k , c ) SVC( z , c ) SV( k ) with adjustments with SCC cover Initialization: compute SCCs { S 0 , . . . , S ℓ } pick k vertices if | S i | ≥ z : pick supportive as supportive vertices vertex for S i as representative uniformly at random map: vertex → representative Updates: re-initialize every c updates Queries: try supportive vertices lookup & use representatives in order of L SV remove invalid entries from map fallback: static graph traversal fallback: mode of SV / SVA Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Single-Source Reachability Subalgorithms Extended Simple Simplified Extended Incremental Algorithm ( SI ) Even-Shiloach Trees ( SES ) Maintains reachability tree: Maintains BFS tree: Insertions: Insertions: extend tree via BFS update BFS levels Deletions: Deletions: reconstruct tree via simplified ES tree routine backward/forward BFS Queries: O (1) time Queries: O (1) time s s v v Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Experiments Alg ora All algorithms implemented in C++17 as part of the open-source algorithms library Algora. libAlgora Code available publicly on Gitlab & Github: libAlgora Algorithms ◮ BFS , DFS , DBFS (DFS-BFS hybrid) ◮ BiBFS (bidirectional BFS) ◮ SV with k = 1, k = 2, k = 3 ⋆ ◮ SVA with k = 1 and c = 1 k , c = 10 k , c = 100 k ⋆ ◮ SVC with z = 25 or z = 50 and c = 10 k , c = 100 k ⋆ ⋆ Fallback: BiBFS ; SSR algorithms: SES , SI [HHS20] Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Experiments: Instances Random dynamic instances ER graphs: n = 100 k and n = 10 m , m init = d · n , d ∈ [1 . 25 . . . 50] σ = 100 k , different ratios of insertions/deletions/queries Stochastic Kronecker graphs with random update sequences: n ≈ 130 k and n ≈ 30 . . . 130 k , m avg = d · n , d = 0 . 7 . . . 16 . 5 σ ± = 1 . 6 m . . . 702 m and σ ± = 282 k . . . 82 m (updates only) Real-world dynamic instances . . . with real-world update sequences: n = 100 k . . . 2 . 2 m , m avg = d · n , d = 5 . 4 . . . 7 . 8 σ ± = 1 . 6 m . . . 86 . 2 m (updates only) . . . with randomized update sequences: n = 31 k . . . 2 . 2 m , m avg = d · n , d = 4 . 7 . . . 10 . 4 σ ± = 1 . 4 m . . . 76 . 4 m (updates only) Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Experiments: Random Instances n = σ = 100 k , ρ IDQ = 1 : 1 : 1 Mean total update time ( s ) DFS / DBFS Mean total query time ( s ) 1 000 SVA(1,1 k ) BFS 100 10 SVA(1,10 k ) BiBFS 10 SVC(25,10 k ) 1 1 SV(1) SV(3) SV(1) SVA SVC(25, ∞ ) 0 . 1 SV(2) SVC 0 . 1 1 . 25 2 . 5 5 10 20 40 1 . 25 2 . 5 5 10 20 40 Density d Density d Mean total operation time ( s ) DFS / DBFS 1 000 10 000 relative to SVC(25, ∞ ) BFS Mean operation time, DFS / DBFS BFS 100 1 000 SVA(1,1 k ) SVA(1,1 k ) SVA(1,10 k ) 10 BiBFS 100 SVA(1,10 k ) BiBFS SVC(25,10 k ) 1 10 SVC(25,10 k ) SV(3) SV(3) SVC(25, ∞ ) SV(1) SV(1) SVC(25, ∞ ) 0 . 1 1 1 . 25 2 . 5 5 10 20 40 1 . 25 2 . 5 5 10 20 40 Density d Density d Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Experiments: Random Instances n = σ = 100 k , ρ IDQ = 1 : 1 : 2 | n = 10 m , σ = 100 k , ρ IDQ = 1 : 1 : 1 Mean total operation time ( s ) Mean total operation time ( s ) SVA(1,1 k ) SVA(1,1 k ) SVA(1,10 k ) 10 1 000 BiBFS BiBFS SVA(1,10 k ) SVC(50,10 k ) 100 1 SVC(25,10 k ) SVC(50, ∞ ) SV(3) SV(3) SV(1) SVC(25, ∞ ) 10 SV(1) 0 . 1 1 . 25 2 . 5 5 10 20 40 1 . 25 2 . 5 5 10 20 40 Density d Density d relative to SVC( 25 , ∞ ) relative to SVC( 50 , ∞ ) Mean operation time, Mean operation time, SVA(1,1 k ) SVA(1,1 k ) 100 BiBFS 100 SVA(1,10 k ) BiBFS SVA(1,10 k ) SVC(25,10 k ) 10 10 SVC(25,10 k ) SV(3) SV(3) SV(1) SV(1) 1 1 SVC(25, ∞ ) SVC(25, ∞ ) 1 . 25 2 . 5 5 10 20 40 1 . 25 2 . 5 5 10 20 40 Density d Density d Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Experiments: Kronecker Instances n ≈ 130 k , σ ± = 1 . 6 m . . . 702 m , ρ UQ = 2 : 1 SVC( 25 , 100k ) SVC( 25 , 10k ) SVA( 1 , 100k ) SVA( 1 , 10k ) SV( 2 ) SV( 1 ) BiBFS 22min 19s 24min 3s Timeout ( 30 min ) 14min 30s 13min 35s 13min 50s 9min 25s 9min 19s 7min 59s 6min 41s 5min 42s 5min 34s 5min 32s 4min 41s 5min 5s 4min 11s 4min 12s 3min 32s 3min 12s 2min 40s 2min 34s 2min 48s 2min 20s 1min 55s 1min 26s 1min 45s 1min 46s 1min 46s 1min 46s 1min 4s 2min 1min 32s 53s 41s 34s 31s 23s answers bio-proteins blog-nat05-6m ca-dblp email-inside gnutella-25 Fastest: SV(1) , SV(2) , SVC(25,100 k ) BFS , DFS , DBFS : > 6 h on ≥ 13 / 20 instances similar picture for n ≈ 30 . . . 130 k Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Experiments: Real-World Instances n = 31 k . . . 2 . 2 m , σ ± = 1 . 6 m . . . 86 . 2 m , ρ UQ = 2 : 1 BiBFS DFS SVC( 50 , 100k ) SVC( 50 , 10k ) SVA( 1 , 100k ) SVA( 1 , 10k ) SV( 2 ) SV( 1 ) 41min Timeout 21h 15h 41min (24h) 13h 9min 8h 29min 7h 40min 6h 54min 6h 57min 5h 24min 4h 44min 4h 44min 4h 42min 4h 25min 3h 46min 3h 24min 3h 30min 2h 16min 2h 31min 2h 10min 1h 47min 59min 36s 1h 43min 1h 42min 53min 29s 50min 22s 46min 52s 51min 14s 2h 1min 1h 12min 37min 43s 30min 50s 1h 18min 42min 41s 22min 26s 28min 22s 17min 46s 1h 1min 11s 30s 24s 37s 23s 14s 31s DE FR IT NL PL SIM Fastest: SV(1) , SV(2) BFS , DFS , DBFS : ≈ 6 % in 24 h on DE instance similar picture on set with randomized updates Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice

Conclusion SV(1) SV(2) 1 SVC(*,100 k ) 2 3 + more stable query time + more stable query time + fast on sparse instances – doubled update time – – considerably increased update time ?? recompute less often Slower by several orders of magnitude : BFS , DFS , DBFS , BiBFS Kathrin Hanauer Faster Fully Dynamic Transitive Closure in Practice