Breaking the Linear-Memory Barrier in Massively Parallel Computing - - PowerPoint PPT Presentation

Breaking the Linear-Memory Barrier in Massively Parallel Computing

MIS on Trees with Strongly Sublinear Memory

Sebastian Brandt, Man anuela la Fis Fischer, Jara Uitto ETH Zurich

Massively Parallel Computing (MPC)

Model:

Massively Parallel Computing (MPC)

Model:

parallel computing framework inspired by MapReduce

Massively Parallel Computing (MPC)

Model:

parallel computing framework inspired by MapReduce

Massively Parallel Computing (MPC)

Model:

Karloff, Suri, Vassilvitskii [SODA’10]

parallel computing framework inspired by MapReduce

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐)

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines Complexity: number of rounds

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines Complexity: number of rounds

Massively Parallel Computing (MPC) Model

Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines Complexity: number of rounds 𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐)

Massively Parallel Computing (MPC) Model

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Synchronous Rounds

• 1. Local Computation

at every machine

• 2. Global Communication

between machines Complexity: number of rounds

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐)

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐)

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐)

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) usual assumption unrealistic ▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11]

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11] for most problems,

• nly direct simulation of

LOCAL/PRAM algorithms known

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11] for most problems,

• nly direct simulation of

LOCAL/PRAM algorithms known

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes. Algorithms have been stuck at this linear-memory barrier!

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11] for most problems,

• nly direct simulation of

LOCAL/PRAM algorithms known

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

Local Memory in MPC

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. usual assumption

• ften unrealistic

▪ ෩ O(𝑜) prohibitively large ▪ sparse graphs trivial Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes. Algorithms have been stuck at this linear-memory barrier! Fundamentally?

• ften trivial

for many problems, admits O(1)-round algorithms based on very simple sampling approach Lattanzi et al. [SPAA’11] for most problems,

• nly direct simulation of

LOCAL/PRAM algorithms known

Breaking the Linear-Memory Barrier:

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier:

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Ghaffari, Kuhn, Uitto [FOCS’19] Conditional Lower Bound Ω log log 𝑜 rounds

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

best we can hope for GKU [FOCS’19]

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Efficient MPC Graph Algorithms with Strongly Sublinear Memory

Breaking the Linear-Memory Barrier: 𝑻 = 𝑷 𝒐𝜺 local memory 𝑵 = 𝑷 𝒏/𝒐𝜺 machines poly lo log lo log 𝒐 rounds Imposed Locality: machines see only subset of nodes, regardless of sparsity of graph Our Approach to Cope with Locality: enhance LOCAL algorithms with global communication ▪ exponentially faster than LOCAL algorithms due to shortcuts ▪ polynomially less memory than most MPC algorithms

Maximal Independent Set (MIS)

Problem:

Maximal Independent Set (MIS)

Maximal Independent Set (MIS)

Maximal Independent Set (MIS)

Maximal Independent Set (MIS)

Independent Set: set of non-adjacent nodes Maximal: no node can be added without violating independence

Maximal Independent Set (MIS)

Independent Set: set of non-adjacent nodes Maximal: no node can be added without violating independence

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Lattanzi et al. [SPAA’11] 𝑃 1 Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Ghaffari et al. [PODC’18] 𝑃(log log 𝑜) Lattanzi et al. [SPAA’11] 𝑃 1 Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Luby’s Algorithm 𝑃(log 𝑜) Ghaffari et al. [PODC’18] 𝑃(log log 𝑜) Lattanzi et al. [SPAA’11] 𝑃 1 Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Luby’s Algorithm 𝑃(log 𝑜) Ghaffari et al. [PODC’18] 𝑃(log log 𝑜) Ghaffari and Uitto [SODA’19] ෨ 𝑃 log 𝑜 Lattanzi et al. [SPAA’11] 𝑃 1 Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art on Trees

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Luby’s Algorithm 𝑃(log 𝑜) Ghaffari et al. [PODC’18] 𝑃(log log 𝑜) Ghaffari and Uitto [SODA’19] ෨ 𝑃 log 𝑜 Lattanzi et al. [SPAA’11] 𝑃 1 Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art on Trees

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Luby’s Algorithm 𝑃(log 𝑜) Ghaffari et al. [PODC’18] 𝑃(log log 𝑜) Ghaffari and Uitto [SODA’19] ෨ 𝑃 log 𝑜 Lattanzi et al. [SPAA’11] 𝑃 1 Trivial solution 𝑃 1 Trivial solution 𝑃 1 Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

෩ Θ(𝑜) ෩ Ω 𝑜1+𝜀 ෩ O 𝑜𝜀

S

MIS: State of the Art on Trees

𝑵 machines 𝑻 memory per machine 𝑵 ⋅ 𝑻 = ෩ 𝑷 (𝒏 + 𝒐) Strongly Sublinear Memory: 𝑇 = ෨ 𝑃 𝑜𝜀 , 0 ≤ 𝜀 < 1 No machine sees all nodes. Linear Memory: 𝑇 = ෨ 𝑃 𝑜 Machines see all nodes. Luby’s Algorithm 𝑃(log 𝑜) Ghaffari et al. [PODC’18] 𝑃(log log 𝑜) Ghaffari and Uitto [SODA’19] ෨ 𝑃 log 𝑜 Lattanzi et al. [SPAA’11] 𝑃 1 Trivial solution 𝑃 1 Our Result 𝑃 log3 log 𝑜 Trivial solution 𝑃 1 Superlinear Memory: 𝑇 = ෨ 𝑃 𝑜1+𝜀 , 0 < 𝜀 ≤ 1 Machines see all nodes.

Our Result

𝑷(𝐦𝐩𝐡𝟒 𝐦𝐩𝐡 𝒐)-round MPC algorithm with 𝐓 = ෩ 𝑷 𝒐𝜺 memory that w.h.p. computes MIS on trees.

Our Result

𝑷(𝐦𝐩𝐡𝟒 𝐦𝐩𝐡 𝒐)-round MPC algorithm with 𝐓 = ෩ 𝑷 𝒐𝜺 memory that w.h.p. computes MIS on trees. ෩ 𝑷 𝐦𝐩𝐡 𝒐 rounds 𝐓 = ෩ 𝑷 𝒐𝜺 memory

Ghaffari and Uitto [SODA’19]

Our Result

𝑷(𝐦𝐩𝐡𝟒 𝐦𝐩𝐡 𝒐)-round MPC algorithm with 𝐓 = ෩ 𝑷 𝒐𝜺 memory that w.h.p. computes MIS on trees. ෩ 𝑷 𝐦𝐩𝐡 𝒐 rounds 𝐓 = ෩ 𝑷 𝒐𝜺 memory

Ghaffari and Uitto [SODA’19]

𝑷 𝐦𝐩𝐡 𝐦𝐩𝐡 𝒐 rounds 𝐓 = ෩ 𝑷 𝒐 memory

Ghaffari et al. [PODC’18]

Our Result

𝑷(𝐦𝐩𝐡𝟒 𝐦𝐩𝐡 𝒐)-round MPC algorithm with 𝐓 = ෩ 𝑷 𝒐𝜺 memory that w.h.p. computes MIS on trees. ෩ 𝑷 𝐦𝐩𝐡 𝒐 rounds 𝐓 = ෩ 𝑷 𝒐𝜺 memory

Ghaffari and Uitto [SODA’19]

𝑷 𝐦𝐩𝐡 𝐦𝐩𝐡 𝒐 rounds 𝐓 = ෩ 𝑷 𝒐 memory

Ghaffari et al. [PODC’18]

Conditional 𝛁(𝐦𝐩𝐡 𝐦𝐩𝐡 𝒐)-round lower bound for 𝐓 = ෩ 𝑷 𝒐𝜺

Ghaffari, Kuhn, and Uitto [FOCS’19]

Algorithm

Algorithm Outline

Algorithm Outline

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

main LOCAL technique Beck [RSA’91]

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

independence due to restriction to trees!

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

independence due to restriction to trees!

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

Non-subsampled High-Degree Node ▪ w.h.p. has many subsampled neighbors ▪ thus w.h.p. has at least one MIS neighbor ▪ hence will be removed from the graph

Subsample-and-Conquer

Polynomial Degree Reduction:

Subsample subsample nodes independently Conquer compute random MIS in subsampled graph ▪ gather connected components ▪ locally compute random 2-coloring ▪ add a color class to MIS

1) Shattering

break graph into small components

i) Degree Reduction ii) LOCAL Shattering Ghaffari [SODA’16]

2) Post-Shattering

solve problem on remaining components

i) Gathering of Components ii) Local Computation

Algorithm Outline

Distributed Union-Find Iterated Subsample-and-Conquer

Conclusion and Open Questions

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees
• ther graph problems?

more general graph families?

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees
• ther graph problems?

more general graph families? MIS & Matching for locally sparse graphs in follow-up work [PODC’19]

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees
• ther graph problems?

more general graph families?

• ther LOCAL techniques?

MIS & Matching for locally sparse graphs in follow-up work [PODC’19]

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees
• ther graph problems?

more general graph families?

• ther LOCAL techniques?
• ther approaches?

MIS & Matching for locally sparse graphs in follow-up work [PODC’19]

MODEL:

Sublinear-Memory MPC 𝑇 = ෨ 𝑃 𝑜𝜀 local memory poly log log 𝑜 rounds

APPROACH:

LOCAL algorithms & global communication

TECHNIQUE:

Shattering

PROBLEM:

MIS

• n trees
• ther graph problems?

more general graph families? MIS & Matching for locally sparse graphs in follow-up work [PODC’19]

• ther LOCAL techniques?

Thank you!

• ther approaches?