Constant-Time Retrieval with Polynomially-small Slack Martin - - PowerPoint PPT Presentation

Constant-Time Retrieval with Polynomially-small Slack

Martin Dietzfelbinger Stefan Walzer September 24, 2018

Motivating Example

The Problem Given a set E ⊆ U of size m and f : E → {0, 1}. Want: Data structure R that, given y ∈ E reproduces f(y). Memory Requirement of R Want: m bits for O . (this work: O m ) Example Data Set: Given names annotated with gender E Ana, Bea, Cal, Dan, Eli, Fen ascii f 1 1 1

1

Motivating Example

The Problem Given a set E ⊆ U of size m and f : E → {0, 1}. Want: Data structure R that, given y ∈ E reproduces f(y). Memory Requirement of R Want: (1 + ε)m bits for ε = O(1). (this work: ε = O(m−δ)) Example Data Set: Given names annotated with gender E Ana, Bea, Cal, Dan, Eli, Fen ascii f 1 1 1

1

Motivating Example

The Problem Given a set E ⊆ U of size m and f : E → {0, 1}. Want: Data structure R that, given y ∈ E reproduces f(y). Memory Requirement of R Want: (1 + ε)m bits for ε = O(1). (this work: ε = O(m−δ)) Example Data Set: Given names annotated with gender E = { Ana, Bea, Cal, Dan, Eli, Fen } ⊆ U = ascii∗ → → → → → → f : 1 1 1

1

Retrieval using Linear Systems

Pick n = (1 + ε)m and h : U → {1, . . . , n}3. Yields A

m n:

Input Hash Values Ana h Ana Bea h Bea Cal h Cal Dan h Dan Eli h Eve Fen h Fen

1 2 3 4 5 6 7 8 9

x

2

Retrieval using Linear Systems

Pick n = (1 + ε)m and h : U → {1, . . . , n}3. Yields A ∈ Fm×n

2

: Input Hash Values Ana h(Ana) = (1, 3, 9) Bea h(Bea) = (2, 3, 4) Cal h(Cal) = (3, 6, 8) Dan h(Dan) = (5, 8, 9) Eli h(Eve) = (2, 8, 9) Fen h(Fen) = (1, 5, 6)         

1

1

2 3

1

4 5 6 7 8 9

1 0 1 1 1 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 0 0 0 1 0 0 1 1 0 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 0 0          x

2

Retrieval using Linear Systems

Pick n = (1 + ε)m and h : U → {1, . . . , n}3. Yields A ∈ Fm×n

2

: Input Hash Values (Ana, 1) h(Ana) = (1, 3, 9) (Bea, 1) h(Bea) = (2, 3, 4) (Cal, 0) h(Cal) = (3, 6, 8) (Dan, 0) h(Dan) = (5, 8, 9) (Eli, 0) h(Eve) = (2, 8, 9) (Fen, 1) h(Fen) = (1, 5, 6)         

1

1

2 3

1

4 5 6 7 8 9

1 0 1 1 1 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 0 0 0 1 0 0 1 1 0 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 0 0          · ⃗ x =          1 1 1         

2

Retrieval using Linear Systems

Pick n = (1 + ε)m and h : U → {1, . . . , n}3. Yields A ∈ Fm×n

2

: Input Hash Values (Ana, 1) h(Ana) = (1, 3, 9) (Bea, 1) h(Bea) = (2, 3, 4) (Cal, 0) h(Cal) = (3, 6, 8) (Dan, 0) h(Dan) = (5, 8, 9) (Eli, 0) h(Eve) = (2, 8, 9) (Fen, 1) h(Fen) = (1, 5, 6)         

1

1

2 3

1

4 5 6 7 8 9

1 0 1 1 1 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 0 0 0 1 0 0 1 1 0 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 0 0          · ⃗ x =          1 1 1          Theorem [DM02] For ε > 0.09 such systems are solvable whp. For ε < 0.09 such systems are not solvable whp. Theorem [Bot08; Mol05; BWZ12] For the system is solvable in O m time whp.

2

Retrieval using Linear Systems

Pick n = (1 + ε)m and h : U → {1, . . . , n}3. Yields A ∈ Fm×n

2

: Input Hash Values (Ana, 1) h(Ana) = (1, 3, 9) (Bea, 1) h(Bea) = (2, 3, 4) (Cal, 0) h(Cal) = (3, 6, 8) (Dan, 0) h(Dan) = (5, 8, 9) (Eli, 0) h(Eve) = (2, 8, 9) (Fen, 1) h(Fen) = (1, 5, 6)         

1

1

2 3

1

4 5 6 7 8 9

1 0 1 1 1 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 0 0 0 1 0 0 1 1 0 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 0 0          · ⃗ x =          1 1 1          Theorem [DM02] For ε > 0.09 such systems are solvable whp. For ε < 0.09 such systems are not solvable whp. Theorem [Bot08; Mol05; BWZ12] For ε > 0.23 the system is solvable in O(m) time whp.

2

Retrieval using Linear Systems

Pick n = (1 + ε)m and h : U → {1, . . . , n}3. Yields A ∈ Fm×n

2

: Input Hash Values (Ana, 1) h(Ana) = (1, 3, 9) (Bea, 1) h(Bea) = (2, 3, 4) (Cal, 0) h(Cal) = (3, 6, 8) (Dan, 0) h(Dan) = (5, 8, 9) (Eli, 0) h(Eve) = (2, 8, 9) (Fen, 1) h(Fen) = (1, 5, 6)         

1

1

2 3

1

4 5 6 7 8 9

1 0 1 1 1 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 0 0 0 1 0 0 1 1 0 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 0 0          ·

                1 1                

=          1 1 1          The retrieval data structure R = (h,⃗ x) query(u) := ∑3

i=1⃗

x[hi(u)] Memory Requirement: ∼ 1.09m bits

(or ∼ 1.23m bits with O(m) construction).

2

Contribution and Previous Work

Paper

memory accesses query

Slack practical? [Maj+96; BPZ13] 3 0.23

• [GOV16]

3 0.09

• [DP08]

k ≈ e−k ✗ [DP08] log m O( log log m

m

) ✗ [Por09] O(1) Θ( log log m √

log m )

✗ ⟨this work⟩ 2 Θ( log m

mα )

• 3

Our Idea: Blocked Linear Systems

Two log-sized Blocks per Line

Pick n = (1 + ε)m, block size ℓ ∈ N and h : U → [n/ℓ] × [n/ℓ] × Fℓ

2 × Fℓ 2.

Hash Values h Ana h Bea h Cal h Dan h Eli h Fen

1 2 3 4

x

4

Two log-sized Blocks per Line

Pick n = (1 + ε)m, block size ℓ ∈ N and h : U → [n/ℓ] × [n/ℓ] × Fℓ

2 × Fℓ 2.

Hash Values h(Ana) = (1, 3, 010, 110) h(Bea) = (1, 4, 111, 110) h(Cal) = (1, 4, 111, 010) h(Dan) = (1, 2, 010, 110) h(Eli) = (2, 4, 111, 001) h(Fen) = (3, 3, 111, 110)         

1

10

2

1

3

10

4

100 110 111 010 010 110 111 001 001          ·⃗ x =          1 1 1         

4

Two log-sized Blocks per Line

Pick n = (1 + ε)m, block size ℓ ∈ N and h : U → [n/ℓ] × [n/ℓ] × Fℓ

2 × Fℓ 2.

Hash Values h(Ana) = (1, 3, 010, 110) h(Bea) = (1, 4, 111, 110) h(Cal) = (1, 4, 111, 010) h(Dan) = (1, 2, 010, 110) h(Eli) = (2, 4, 111, 001) h(Fen) = (3, 3, 111, 110)         

1

10

2

1

3

10

4

100 110 111 010 010 110 111 001 001          ·⃗ x =          1 1 1          Theorem For a block size of ℓ = 4 log m such systems are solvable whp, provided that ε ≥ 27 log(m)/m.

4

Two log-sized Blocks per Line

Pick n = (1 + ε)m, block size ℓ ∈ N and h : U → [n/ℓ] × [n/ℓ] × Fℓ

2 × Fℓ 2.

Hash Values h(Ana) = (1, 3, 010, 110) h(Bea) = (1, 4, 111, 110) h(Cal) = (1, 4, 111, 010) h(Dan) = (1, 2, 010, 110) h(Eli) = (2, 4, 111, 001) h(Fen) = (3, 3, 111, 110)         

1

10

2

1

3

10

4

100 110 111 010 010 110 111 001 001          ·

                 1 1 1 1 1 1                 

=          1 1 1          The retrieval data structure R = (h,⃗ x) query(u) := ⟨p1, x[b1]⟩ ⊕ ⟨p2, x[b2]⟩

//O(1)

where h(u) = (b1, b2, p1, p2) and x[b] is the b-th block of x Memory Requirement: (1 + O( log m

m ))m bits. 4

Efficient Construction

Contribution and Previous Work

Paper

memory accesses query

Slack tconstruct [Maj+96; BPZ13] 3 0.23 O(m) [DP08] k ≈ e−k O(m3) O mC

w

[DP08] log m O( log log m

m

) O(m3) [Por09] O(1) Ω( log log m √

log m )

O(m) ⟨this paper⟩ 2 Θ( log m

m )

O(m3) O

mC w log C

Input partitioned into chunks of size C. Word size w.

5

Contribution and Previous Work

Paper

memory accesses query

Slack tconstruct [Maj+96; BPZ13] 3 0.23 O(m) [DP08][ADR09; GOV16] k ≈ e−k+ log m

C

O(m3) O( mC2

w )

[DP08] log m O( log log m

m

) O(m3) [Por09] O(1) Ω( log log m √

log m )

O(m) ⟨this paper⟩ 2 Θ( log m

mC )

O(m3) O( mC2

w log C)

Input partitioned into chunks of size C. Word size w.

5

Speed-Up (i): Partition the Input into Chunks

Construct chunked retrieval data structure for E

• Pick chunksize C and random h

m C .

• Partition E into E

E m

C using h.

• Construct retrieval data structure Ri for Ei,

i

m C .

• query u is redirected to Rh u

Effects on construction time and memory

• Construction time: m

C

C mC

(using Gaussian elimination).

• Additional memory requirement: m

C pointers of size log m bits. log m C 6

Speed-Up (i): Partition the Input into Chunks

Construct chunked retrieval data structure for E

• Pick chunksize C and random h : U → [ m

C ].

• Partition E into E

E m

C using h.

• Construct retrieval data structure Ri for Ei,

i

m C .

• query u is redirected to Rh u

Effects on construction time and memory

• Construction time: m

C

C mC

(using Gaussian elimination).

• Additional memory requirement: m

C pointers of size log m bits. log m C 6

Speed-Up (i): Partition the Input into Chunks

Construct chunked retrieval data structure for E

• Pick chunksize C and random h : U → [ m

C ].

• Partition E into E1, . . . , E m

C using h.

• Construct retrieval data structure Ri for Ei,

i

m C .

• query u is redirected to Rh u

Effects on construction time and memory

• Construction time: m

C

C mC

(using Gaussian elimination).

• Additional memory requirement: m

C pointers of size log m bits. log m C 6

Speed-Up (i): Partition the Input into Chunks

Construct chunked retrieval data structure for E

• Pick chunksize C and random h : U → [ m

C ].

• Partition E into E1, . . . , E m

C using h.

• Construct retrieval data structure Ri for Ei, (1 ≤ i ≤ m

C ).

• query u is redirected to Rh u

Effects on construction time and memory

• Construction time: m

C

C mC

(using Gaussian elimination).

• Additional memory requirement: m

C pointers of size log m bits. log m C 6

Speed-Up (i): Partition the Input into Chunks

Construct chunked retrieval data structure for E

• Pick chunksize C and random h : U → [ m

C ].

• Partition E into E1, . . . , E m

C using h.

• Construct retrieval data structure Ri for Ei, (1 ≤ i ≤ m

C ).

• query(u) is redirected to Rh(u)

Effects on construction time and memory

• Construction time: m

C

C mC

(using Gaussian elimination).

• Additional memory requirement: m

C pointers of size log m bits. log m C 6

Speed-Up (i): Partition the Input into Chunks

Construct chunked retrieval data structure for E

• Pick chunksize C and random h : U → [ m

C ].

• Partition E into E1, . . . , E m

C using h.

• Construct retrieval data structure Ri for Ei, (1 ≤ i ≤ m

C ).

• query(u) is redirected to Rh(u)

Effects on construction time and memory

• Construction time: m

C · C3 = mC2 (using Gaussian elimination).

• Additional memory requirement: m

C pointers of size log m bits.

⇒ ε′ = ε + log m

C 6

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1111 1 1111 1011 2 0010 1111 3 1001 0011 1 4 0101 1111 5 1011 0111 1 6 0101 1011 7 1011 1001 1 8 1111 1100 9 0110 0100 1 10 0011 0001 1 11 1011 0100 12 1011 0010 1 13 1000 1101 14 0100 0111 15 1001 1001 1 16 1010 0010 17 1111 0010 1 18 0010 1100 19 1001 0111 20 1000 1101 1 21 1011 1110 1 22 0110 1 23 0011 1111 1 24 1001 1101 1 25 0001 26 1110 1000 27 0110 1010 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1111 1 1111 1011 2 0010 1111 3 1001 0011 1 4 0101 1111 5 1011 0111 1 6 0101 1011 7 1011 1001 1 8 1111 1100 9 0110 0100 1 10 0011 0001 1 11 1011 0100 12 1011 0010 1 13 1000 1101 14 0100 0111 15 1001 1001 1 16 1010 0010 17 1111 0010 1 18 0010 1100 19 1001 0111 20 1000 1101 1 21 1011 1110 1 22 0110 1 23 0011 1111 1 24 1001 1101 1 25 0001 26 1110 1000 27 0110 1010 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1111 1 1111 1011 2 0010 1111 3 1001 001 1 4 0101 1111 5 1011 0111 1 6 0101 1011 7 101 1001 1 8 1111 1100 9 011 0100 1 10 001 0001 1 11 1011 0100 12 1011 0010 1 13 1000 1101 14 010 0111 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 1000 1101 1 21 1011 1110 1 22 0110 1 23 0011 1111 1 24 1001 1101 1 26 111 1000 27 0110 1010 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1111 1 1111 1011 2 0010 1111 3 1001 001 1 4 0101 1111 5 1011 0111 1 6 0101 1011 7 101 1001 1 8 1111 1100 9 011 0100 1 10 001 0001 1 11 1011 0100 12 1011 0010 1 13 1000 1101 14 010 0111 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 1000 1101 1 21 1011 1110 1 22 0110 1 23 0011 1111 1 24 1001 1101 1 26 111 1000 27 0110 1010 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1111 1 101 1011 1 2 0010 1111 3 1001 001 1 4 0101 1111 5 1011 0111 1 6 0101 1011 7 101 101 1 8 101 1100 1 9 011 0100 1 10 001 001 1 11 1011 0100 12 111 0010 13 1000 111 14 010 001 1 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 100 1101 1 21 1011 1110 1 23 011 1111 24 1001 1101 1 26 111 1000 27 1010 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1111 1 101 1011 1 2 0010 1111 3 1001 001 1 4 0101 1111 5 1011 0111 1 6 0101 1011 7 101 101 1 8 101 1100 1 9 011 0100 1 10 001 001 1 11 1011 0100 12 111 0010 13 1000 111 14 010 001 1 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 100 1101 1 21 1011 1110 1 23 011 1111 24 1001 1101 1 26 111 1000 27 1010 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 011 1 101 1011 1 2 100 1111 3 1001 001 1 4 0101 1111 5 1011 111 1 6 0101 1011 7 101 101 1 8 101 1100 1 9 011 0100 1 10 001 001 1 11 1011 0100 12 111 0010 13 1000 111 14 010 001 1 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 100 1101 1 21 1011 1110 1 23 011 1111 24 1001 1101 1 26 111 100 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 011 1 101 1011 1 2 100 1111 3 1001 001 1 4 0101 1111 5 1011 111 1 6 0101 1011 7 101 101 1 8 101 1100 1 9 011 0100 1 10 001 001 1 11 1011 0100 12 111 0010 13 1000 111 14 010 001 1 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 100 1101 1 21 1011 1110 1 23 011 1111 24 1001 1101 1 26 111 100 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1 101 1011 1 2 10 1111 3 1001 001 1 4 0101 1111 5 1011 10 1 6 0101 1011 7 101 101 1 8 101 1100 1 9 011 0100 1 10 001 001 1 11 1011 0100 12 111 0010 13 1000 111 14 010 001 1 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 100 1101 1 21 1011 1110 1 23 011 1111 24 1001 1101 1 26 111 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 3 4 5 6 7 rhs 1 101 1011 1 2 10 1111 3 1001 001 1 4 0101 1111 5 1011 10 1 6 0101 1011 7 101 101 1 8 101 1100 1 9 011 0100 1 10 001 001 1 11 1011 0100 12 111 0010 13 1000 111 14 010 001 1 15 1001 1001 1 16 1010 001 17 1111 0010 1 18 0010 1100 19 1001 0111 20 100 1101 1 21 1011 1110 1 23 011 1111 24 1001 1101 1 26 111 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 6 7 rhs 1 101 1011 1 2 000 10 1111 3 000 1001 001 1 4 000 0101 1111 5 000 1011 10 1 6 000 0101 1011 7 101 101 1 8 101 1100 1 9 000 011 0100 1 10 100 001 1 11 000 1011 0100 12 111 0010 13 111 1000 14 100 010 1 15 000 1001 1001 1 16 000 1010 001 17 000 1111 0010 1 18 000 0010 1100 19 000 1001 0111 20 001 1101 1 21 000 1011 1110 1 23 110 1111 24 000 1001 1101 1 26 000 111 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 6 7 rhs 1 101 1011 1 2 000 10 1111 3 000 1001 001 1 4 000 0101 1111 5 000 1011 10 1 6 000 0101 1011 7 101 101 1 8 101 1100 1 9 000 011 0100 1 10 100 001 1 11 000 1011 0100 12 111 0010 13 111 1000 14 100 010 1 15 000 1001 1001 1 16 000 1010 001 17 000 1111 0010 1 18 000 0010 1100 19 000 1001 0111 20 001 1101 1 21 000 1011 1110 1 23 110 1111 24 000 1001 1101 1 26 000 111 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 6 7 rhs 1 101 1011 1 2 110 10 3 000 1001 001 1 4 000 0101 1111 5 000 1011 10 1 6 000 0101 1011 7 101 101 1 8 101 110 1 9 000 011 0100 1 10 100 001 1 11 000 1011 0100 12 111 0010 13 111 1000 14 100 010 1 15 000 1001 1001 1 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 000 1001 0111 20 111 001 1 21 000 1011 111 1 24 110 1001 001 1 26 000 111 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 6 7 rhs 1 101 1011 1 2 110 10 3 000 1001 001 1 4 000 0101 1111 5 000 1011 10 1 6 000 0101 1011 7 101 101 1 8 101 110 1 9 000 011 0100 1 10 100 001 1 11 000 1011 0100 12 111 0010 13 111 1000 14 100 010 1 15 000 1001 1001 1 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 000 1001 0111 20 111 001 1 21 000 1011 111 1 24 110 1001 001 1 26 000 111 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 000 1001 001 1 4 000 0101 1111 5 110 1011 1 6 000 0101 1011 7 101 101 1 8 101 110 1 9 000 011 0100 1 10 100 001 1 11 000 1011 0100 12 111 0010 13 111 1000 14 100 010 1 15 000 1001 1001 1 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 000 1001 0111 20 111 001 1 21 000 1011 111 1 24 110 1001 001 1 26 110 111 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 000 1001 001 1 4 000 0101 1111 5 110 1011 1 6 000 0101 1011 7 101 101 1 8 101 110 1 9 000 011 0100 1 10 100 001 1 11 000 1011 0100 12 111 0010 13 111 1000 14 100 010 1 15 000 1001 1001 1 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 000 1001 0111 20 111 001 1 21 000 1011 111 1 24 110 1001 001 1 26 110 111 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 000 1001 001 1 4 110 111 1111 1 6 110 111 1011 1 7 101 101 1 8 101 110 1 9 000 011 0100 1 10 100 001 1 11 110 0100 1 12 111 0010 13 111 1000 14 100 010 1 15 110 001 1001 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 110 1001 110 1 20 111 001 1 21 110 111 24 110 1001 001 1 26 110 111 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 000 1001 001 1 4 110 111 1111 1 6 110 111 1011 1 7 101 101 1 8 101 110 1 9 000 011 0100 1 10 100 001 1 11 110 0100 1 12 111 0010 13 111 1000 14 100 010 1 15 110 001 1001 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 110 1001 110 1 20 111 001 1 21 110 111 24 110 1001 001 1 26 110 111 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 000 1001 001 1 4 110 111 1111 1 6 110 111 111 1 7 101 101 1 8 101 110 1 9 110 011 10 100 001 1 12 111 0010 13 111 1000 14 100 010 1 15 110 001 101 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 110 1001 110 1 20 111 001 1 21 110 111 24 110 101 001 1 26 110 111 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 000 1001 001 1 4 110 111 1111 1 6 110 111 111 1 7 101 101 1 8 101 110 1 9 110 011 10 100 001 1 12 111 0010 13 111 1000 14 100 010 1 15 110 001 101 16 000 1010 001 17 000 1111 001 1 18 000 0010 1100 19 110 1001 110 1 20 111 001 1 21 110 111 24 110 101 001 1 26 110 111 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 110 1001 01 1 4 110 111 1111 1 6 110 111 111 1 7 011 11 1 8 101 110 1 10 010 01 1 12 111 0010 13 111 1000 14 100 01 1 15 110 001 101 16 110 1010 01 17 000 1111 001 1 18 000 0010 1100 19 110 1001 110 1 20 111 001 1 21 110 111 24 110 101 001 1 26 000 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 110 1001 01 1 4 110 111 1111 1 6 110 111 111 1 7 011 11 1 8 101 110 1 10 010 01 1 12 111 0010 13 111 1000 14 100 01 1 15 110 001 101 16 110 1010 01 17 000 1111 001 1 18 000 0010 1100 19 110 1001 110 1 20 111 001 1 21 110 111 24 110 101 001 1 26 000 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 110 1001 01 1 4 110 111 1111 1 6 110 111 111 1 7 011 11 1 8 101 11 1 10 010 01 1 12 111 0010 13 111 1000 14 100 01 1 15 110 001 101 16 110 1010 01 17 110 1111 11 1 18 000 0010 1100 19 110 1001 110 1 20 001 11 1 24 000 101 11 1 26 000 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 110 1001 01 1 4 110 111 1111 1 6 110 111 111 1 7 011 11 1 8 101 11 1 10 010 01 1 12 111 0010 13 111 1000 14 100 01 1 15 110 001 101 16 110 1010 01 17 110 1111 11 1 18 000 0010 1100 19 110 1001 110 1 20 001 11 1 24 000 101 11 1 26 000 10 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 110 1001 1 1 4 110 111 1111 1 6 110 111 111 1 7 011 1 1 8 101 11 1 10 010 1 1 12 111 0010 13 111 1000 14 100 1 1 15 110 001 101 16 110 1010 1 17 110 1111 11 1 18 000 0010 1100 19 110 1001 110 1 20 001 11 1 24 000 101 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 7 rhs 1 101 1011 1 3 110 1001 1 1 4 110 111 1111 1 6 110 111 111 1 7 011 1 1 8 101 11 1 10 010 1 1 12 111 0010 13 111 1000 14 100 1 1 15 110 001 101 16 110 1010 1 17 110 1111 11 1 18 000 0010 1100 19 110 1001 110 1 20 001 11 1 24 000 101 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 rhs 8 100 1 101 1011 1 3 110 1001 1 1 4 110 111 1111 1 6 110 111 111 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 1000 14 100 1 1 15 110 001 101 16 110 1010 1 17 111 1111 18 000 0010 1100 19 110 1001 110 1 24 001 101 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 1 2 4 5 rhs 8 100 1 101 1011 1 3 110 1001 1 1 4 110 111 1111 1 6 110 111 111 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 1000 14 100 1 1 15 110 001 101 16 110 1010 1 17 111 1111 18 000 0010 1100 19 110 1001 110 1 24 001 101 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 4 5 rhs 8 100 1 101 1011 1 3 001 011 1 1 4 110 111 1111 1 6 110 111 111 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 100 14 100 1 1 15 110 001 101 16 110 101 1 18 000 001 1100 19 001 011 110 1 24 001 101 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 4 5 rhs 8 100 1 101 1011 1 3 001 011 1 1 4 110 111 1111 1 6 110 111 111 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 100 14 100 1 1 15 110 001 101 16 110 101 1 18 000 001 1100 19 001 011 110 1 24 001 101 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 4 5 rhs 8 100 1 101 1011 1 3 001 011 1 1 4 110 111 1111 1 6 111 111 01 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 100 14 100 1 1 15 111 001 16 110 101 1 18 000 001 1100 19 001 011 110 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 4 5 rhs 8 100 1 101 1011 1 3 001 011 1 1 4 110 111 1111 1 6 111 111 01 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 100 14 100 1 1 15 111 001 16 110 101 1 18 000 001 1100 19 001 011 110 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 4 5 rhs 8 100 1 101 1011 1 3 001 011 1 1 4 001 11 1111 1 6 000 11 01 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 100 14 100 1 1 16 110 101 1 18 000 001 1100 19 001 011 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 2 4 5 rhs 8 100 1 101 1011 1 3 001 011 1 1 4 001 11 1111 1 6 000 11 01 1 7 011 1 1 10 010 1 1 12 111 0010 13 111 100 14 100 1 1 16 110 101 1 18 000 001 1100 19 001 011 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 3 101 011 4 001 11 1111 1 6 000 11 01 1 12 111 0010 13 111 100 16 010 101 1 18 000 001 1100 19 001 011 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 3 101 011 4 001 11 1111 1 6 000 11 01 1 12 111 0010 13 111 100 16 010 101 1 18 000 001 1100 19 001 011 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 3 111 11 1 4 001 11 1111 1 6 000 11 01 1 12 111 0010 13 111 10 18 010 10 1100 1 19 011 11 11 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 3 111 11 1 4 001 11 1111 1 6 000 11 01 1 12 111 0010 13 111 10 18 010 10 1100 1 19 011 11 11 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 4 001 11 1111 1 6 000 11 01 1 12 111 0010 13 111 1 18 010 1 1100 1 19 100 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 1 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 4 001 11 1111 1 6 000 11 01 1 12 111 0010 13 111 1 18 010 1 1100 1 19 100 11 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 4 101 1111 6 100 01 12 111 0010 13 111 1 18 010 1 1100 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 4 5 rhs 7 111 8 100 10 110 1 101 1011 1 4 101 1111 6 100 01 12 111 0010 13 111 1 18 010 1 1100 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 4 rhs 7 111 8 100 10 110 1 101 1011 1 4 101 1111 12 111 0010 13 111 1 18 010 1 1100 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 4 rhs 7 111 8 100 10 110 1 101 1011 1 4 101 1111 12 111 0010 13 111 1 18 010 1 1100 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 4 rhs 7 111 8 100 10 110 1 000 010 1 12 111 001 13 111 1 18 010 1 110 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 0 4 rhs 7 111 8 100 10 110 1 000 010 1 12 111 001 13 111 1 18 010 1 110 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 4 rhs 7 111 8 100 10 110 1 000 010 1 12 111 001 18 101 110 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 4 rhs 7 111 8 100 10 110 1 000 010 1 12 111 001 18 101 110 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 4 rhs 7 111 8 100 10 110 1 101 10 12 111 01 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 4 rhs 7 111 8 100 10 110 1 101 10 12 111 01 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 4 rhs 7 111 8 100 10 110 1 101 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D 4 rhs 7 111 8 100 10 110 1 101 1 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D rhs 7 111 8 100 10 110 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss (adapted from [GOV16])

D rhs 7 111 8 100 10 110 If some equation e involves exactly 1 sparse block: Pick variable v

• ccuring

in e, eliminate v from all other

• equations. Remove e and v.

Otherwise: Pick sparse block with largest ratio of

incident equations variables

. Make the block dense.

7

Speed-Up (ii): Blocked Lazy Gauss – Experiments

10000 20000 30000 40000 50000 60000 0.05 0.1 0.15 0.2

fraction of remaining variables system size n (ℓ = 16 constant)

5 10 15 20 25 30 0.05 0.1 0.15

fraction of remaining variables block size ℓ (n = 104 constant)

For ℓ = 16: Yields Speed-Up of ∼

1 0.153 ≈ 300

(if used before Gaussian elimination)

8

Speed-Up (iii): Method of Four Russians

a1 : a2 : a3 : a4 : a5 : a6 : a7 : a8 : a9 : a10 :          1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 0 0         

a a a a a a a a a a a a

Rough Idea

• Choose k

log n. Assume

k k is regular.

• Compute all sums T

c a ck ak c ck .

• For each p

k there is vp

T starting with p.

• For each k

i n: Add vai

k to ai

becomes

n k k.

• k

n k n row additions handle k log n columns. Speed-up of log n.

9

Speed-Up (iii): Method of Four Russians

a1 : a2 : a3 : a4 : a5 : a6 : a7 : a8 : a9 : a10 :          1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 0 0         

a a a a a a a a a a a a

k = log n Rough Idea

• Choose k = log n.

Assume

k k is regular.

• Compute all sums T

c a ck ak c ck .

• For each p

k there is vp

T starting with p.

• For each k

i n: Add vai

k to ai

becomes

n k k.

• k

n k n row additions handle k log n columns. Speed-up of log n.

9

Speed-Up (iii): Method of Four Russians

a1 : a2 : a3 : a4 : a5 : a6 : a7 : a8 : a9 : a10 :          1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 0 0         

a a a a a a a a a a a a

k = log n Rough Idea

• Choose k = log n. Assume

∈ {0, 1}k×k is regular.

• Compute all sums T

c a ck ak c ck .

• For each p

k there is vp

T starting with p.

• For each k

i n: Add vai

k to ai

becomes

n k k.

• k

n k n row additions handle k log n columns. Speed-up of log n.

9

Speed-Up (iii): Method of Four Russians

a1 : a2 : a3 : a4 : a5 : a6 : a7 : a8 : a9 : a10 :          1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 0 0         

⃗ a1 a2 a3 a1⊕a2 a1⊕a3 a2⊕a3 a1⊕a2⊕a3

k = log n Rough Idea

• Choose k = log n. Assume

∈ {0, 1}k×k is regular.

• Compute all sums T = {c1 · a1 ⊕ . . . ⊕ ck · ak | c1, . . . , ck ∈ {0, 1}}.
• For each p

k there is vp

T starting with p.

• For each k

i n: Add vai

k to ai

becomes

n k k.

• k

n k n row additions handle k log n columns. Speed-up of log n.

9

Speed-Up (iii): Method of Four Russians

a1 : a2 : a3 : a4 : a5 : a6 : a7 : a8 : a9 : a10 :          1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 0 0         

⃗ a1 a2 a3 a1⊕a2 a1⊕a3 a2⊕a3 a1⊕a2⊕a3

k = log n Rough Idea

• Choose k = log n. Assume

∈ {0, 1}k×k is regular.

• Compute all sums T = {c1 · a1 ⊕ . . . ⊕ ck · ak | c1, . . . , ck ∈ {0, 1}}.
• For each p ∈ {0, 1}k there is vp ∈ T starting with p.
• For each k

i n: Add vai

k to ai

becomes

n k k.

• k

n k n row additions handle k log n columns. Speed-up of log n.

9

Speed-Up (iii): Method of Four Russians

a1 : a2 : a3 : a4 : a5 : a6 : a7 : a8 : a9 : a10 :          1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 0 0         

⃗ a1 a2 a3 a1⊕a2 a1⊕a3 a2⊕a3 a1⊕a2⊕a3

k = log n Rough Idea

• Choose k = log n. Assume

∈ {0, 1}k×k is regular.

• Compute all sums T = {c1 · a1 ⊕ . . . ⊕ ck · ak | c1, . . . , ck ∈ {0, 1}}.
• For each p ∈ {0, 1}k there is vp ∈ T starting with p.
• For each k < i ≤ n: Add vai[1...k] to ai
• becomes 0(n−k)×k.
• k

n k n row additions handle k log n columns. Speed-up of log n.

9

Speed-Up (iii): Method of Four Russians

a1 : a2 : a3 : a4 : a5 : a6 : a7 : a8 : a9 : a10 :          1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 0 0         

⃗ a1 a2 a3 a1⊕a2 a1⊕a3 a2⊕a3 a1⊕a2⊕a3

k = log n Rough Idea

• Choose k = log n. Assume

∈ {0, 1}k×k is regular.

• Compute all sums T = {c1 · a1 ⊕ . . . ⊕ ck · ak | c1, . . . , ck ∈ {0, 1}}.
• For each p ∈ {0, 1}k there is vp ∈ T starting with p.
• For each k < i ≤ n: Add vai[1...k] to ai
• becomes 0(n−k)×k.
• 2k + (n − k) < 2n row additions handle k = log n columns.

⇒ Speed-up of log n.

9

Speed-Up (iv): Word-Level Parallelism

Adding vectors For x, y ∈ {0, 1}n a Word-RAM can compute x ⊕ y in time O(n/w). In practice, this is a speed-up of w = 64.

10

Contribution and Previous Work

Paper tquery Slack tconstruct [Maj+96; BPZ13] 3 0.23 O(m) [GOV16] k ≈ e−k + log m

C

O( mC2

w )

[DP08] log m O( log log m

m

) O(m3) [Por09] O(1) Ω( log log m √

log m )

O(m) ⟨this paper⟩ 2 Θ( log m

C )

O( mC2

w log C) 11

Experiments

Experiments

Slack Construction [µs/key] Lookup [ns] [GOV16] k = 3 9% 1.12 210 [GOV16] k = 4 3% 1.75 236 ⟨this paper⟩ 0.24% 2.6 75-125

10000 20000 30000 40000 50000 60000 5 10 15

time in [µs/key] chunk size C

: BlockedLazyGauss : 4 Russians : sum

12

Experiments

Slack Construction [µs/key] Lookup [ns] [GOV16] k = 3 9% 1.12 210 [GOV16] k = 4 3% 1.75 236 ⟨this paper⟩ 0.24% 2.6 75-125

10000 20000 30000 40000 50000 60000 5 10 15

time in [µs/key] chunk size C

ℓ = 16 : BlockedLazyGauss : 4 Russians : sum

12

Space Requirement (backup slide)

Number of bits bits used for per element m entropy lower bound 1.000000 εm intended inner slack 0.000500 ∑

i ni − (1 + ε)m

padding so ℓ | ni 0.000716 ⌈log maxi si⌉ · m/C seed for each chunk 0.000200 ⌈log maxi di⌉ · m/C

• ffset info for each chunk

0.000900 [not discussed] various global counters 0.000062 all of the above 1.002378

• m = 107, chunk size C = 104, ℓ = 16, ε = 0.0005,
• ni is actual size of i-th chunk (rounded to multiple of ℓ),
• si is the seed of the i-th chunk (number of failed constructions),
• di is oi − E[oi] where oi = ∑

j<i ni/ℓ.

13

References: i

References

Martin Aumüller, Martin Dietzfelbinger and Michael Rink. ‘Experimental Variations

• f a Theoretically Good Retrieval Data Structure’. In: Proc. 17th ESA. 2009. doi:

10.1007/978-3-642-04128-0_66. Fabiano Cupertino Botelho. ‘Near-Optimal Space Perfect Hashing Algorithms’. PhD thesis. Federal University of Minas Gerais, 2008. url: http://homepages.dcc.ufmg.br/~fbotelho/en/pub/thesis.pdf. Fabiano Cupertino Botelho, Rasmus Pagh and Nivio Ziviani. ‘Practical Perfect Hashing in Nearly Optimal Space’. In: Inf. Syst. (2013). doi: 10.1016/j.is.2012.06.002. Fabiano Cupertino Botelho, Nicholas C. Wormald and Nivio Ziviani. ‘Cores of random r-partite hypergraphs’. In: Inf. Process. Lett. (2012). doi: 10.1016/j.ipl.2011.10.017.

References: ii

Olivier Dubois and Jacques Mandler. ‘The 3-XORSAT Threshold’. In: Proc. 43rd FOCS.

• 2002. doi: 10.1109/SFCS.2002.1182002.

Martin Dietzfelbinger and Rasmus Pagh. ‘Succinct Data Structures for Retrieval and Approximate Membership (Extended Abstract)’. In: Proc. 35th ICALP (1). 2008. doi: 10.1007/978-3-540-70575-8_32. Marco Genuzio, Giuseppe Ottaviano and Sebastiano Vigna. ‘Fast Scalable Construction of (Minimal Perfect Hash) Functions’. In: Experimental Algorithms - 15th International Symposium, SEA 2016, St. Petersburg, Russia, June 5-8, 2016,

• Proceedings. 2016, pp. 339–352. doi: 10.1007/978-3-319-38851-9_23. url:

https://doi.org/10.1007/978-3-319-38851-9_23. Bohdan S. Majewski, Nicholas C. Wormald, George Havas and Zbigniew J. Czech. ‘A Family of Perfect Hashing Methods’. In: Comput. J. (1996). doi: 10.1093/comjnl/39.6.547. Michael Molloy. ‘Cores in random hypergraphs and Boolean formulas’. In: Random

• Struct. Algorithms 27.1 (2005), pp. 124–135. doi: 10.1002/rsa.20061.

Ely Porat. ‘An Optimal Bloom Filter Replacement Based on Matrix Solving’. In: Proc. 4th CSR. 2009. doi: 10.1007/978-3-642-03351-3_25.