Constant-Time Retrieval with Polynomially-small Slack Martin - PowerPoint PPT Presentation

Constant-Time Retrieval with Polynomially-small Slack Martin Dietzfelbinger Stefan Walzer September 24, 2018

Motivating Example Cal , 1 0 0 0 1 1 f ascii Fen Eli , Dan , Bea , The Problem Ana , E Example Data Set: Given names annotated with gender ) O m . (this work: O m bits for Want: Memory Requirement of R 1 Given a set E ⊆ U of size m and f : E → { 0 , 1 } . Want: Data structure R that, given y ∈ E reproduces f ( y ) .

Motivating Example Eli , 1 0 0 0 1 1 f ascii Fen Dan , The Problem Cal , Bea , Ana , E Example Data Set: Given names annotated with gender Memory Requirement of R 1 Given a set E ⊆ U of size m and f : E → { 0 , 1 } . Want: Data structure R that, given y ∈ E reproduces f ( y ) . Want: (1 + ε ) m bits for ε = O (1) . (this work: ε = O ( m − δ ) )

Motivating Example Eli , 1 0 0 0 1 1 The Problem Fen 1 Dan , Bea , Ana , Memory Requirement of R Example Data Set: Given names annotated with gender Cal , Given a set E ⊆ U of size m and f : E → { 0 , 1 } . Want: Data structure R that, given y ∈ E reproduces f ( y ) . Want: (1 + ε ) m bits for ε = O (1) . (this work: ε = O ( m − δ ) ) E = { } ⊆ U = ascii ∗ �→ �→ �→ �→ �→ �→ f :

Retrieval using Linear Systems h Eve x 9 8 7 6 5 4 3 2 1 h Fen Fen Eli h Dan Dan h Cal Cal h Bea Bea h Ana Ana Hash Values Input n : m Yields A 2 Pick n = (1 + ε ) m and h : U → { 1 , . . . , n } 3 .

Retrieval using Linear Systems Eli x 9 8 7 6 5 4 3 2 1 Fen 2 Ana Dan : Input Hash Values Bea Cal Pick n = (1 + ε ) m and h : U → { 1 , . . . , n } 3 . Yields A ∈ F m × n 2 h ( Ana ) = (1 , 3 , 9)  1 0 1 0 0 0 0 0 1  h ( Bea ) = (2 , 3 , 4) 0 1 1 1 0 0 0 0 0     h ( Cal ) = (3 , 6 , 8) 0 0 1 0 0 1 0 1 0     0 0 0 0 1 0 0 1 1 h ( Dan ) = (5 , 8 , 9)       h ( Eve ) = (2 , 8 , 9) 0 1 0 0 0 0 0 1 1   h ( Fen ) = (1 , 5 , 6) 1 0 0 0 1 1 0 0 0

Retrieval using Linear Systems 1 8 7 6 5 4 3 2 9 2 Hash Values : Input Pick n = (1 + ε ) m and h : U → { 1 , . . . , n } 3 . Yields A ∈ F m × n 2 ( Ana , 1) h ( Ana ) = (1 , 3 , 9)  1 0 1 0 0 0 0 0 1   1  ( Bea , 1) h ( Bea ) = (2 , 3 , 4) 0 1 1 1 0 0 0 0 0 1         ( Cal , 0) h ( Cal ) = (3 , 6 , 8) 0 0 1 0 0 1 0 1 0 0     · ⃗ x =     0 0 0 0 1 0 0 1 1 0 ( Dan , 0) h ( Dan ) = (5 , 8 , 9)             ( Eli , 0) h ( Eve ) = (2 , 8 , 9) 0 1 0 0 0 0 0 1 1 0     ( Fen , 1) h ( Fen ) = (1 , 5 , 6) 1 0 0 0 1 1 0 0 0 1

Retrieval using Linear Systems 1 the system is solvable in O m time whp. For Theorem [Bot08; Mol05; BWZ12] Theorem [DM02] 9 8 7 6 5 4 3 2 2 Input : Hash Values Pick n = (1 + ε ) m and h : U → { 1 , . . . , n } 3 . Yields A ∈ F m × n 2 ( Ana , 1) h ( Ana ) = (1 , 3 , 9)  1 0 1 0 0 0 0 0 1   1  ( Bea , 1) h ( Bea ) = (2 , 3 , 4) 0 1 1 1 0 0 0 0 0 1         ( Cal , 0) h ( Cal ) = (3 , 6 , 8) 0 0 1 0 0 1 0 1 0 0     · ⃗ x =     0 0 0 0 1 0 0 1 1 0 ( Dan , 0) h ( Dan ) = (5 , 8 , 9)             ( Eli , 0) h ( Eve ) = (2 , 8 , 9) 0 1 0 0 0 0 0 1 1 0     ( Fen , 1) h ( Fen ) = (1 , 5 , 6) 1 0 0 0 1 1 0 0 0 1 For ε > 0 . 09 such systems are solvable whp. For ε < 0 . 09 such systems are not solvable whp.

Retrieval using Linear Systems 1 Theorem [Bot08; Mol05; BWZ12] Theorem [DM02] 9 8 7 6 5 4 3 2 2 Input : Hash Values Pick n = (1 + ε ) m and h : U → { 1 , . . . , n } 3 . Yields A ∈ F m × n 2 ( Ana , 1) h ( Ana ) = (1 , 3 , 9)  1 0 1 0 0 0 0 0 1   1  ( Bea , 1) h ( Bea ) = (2 , 3 , 4) 0 1 1 1 0 0 0 0 0 1         ( Cal , 0) h ( Cal ) = (3 , 6 , 8) 0 0 1 0 0 1 0 1 0 0     · ⃗ x =     0 0 0 0 1 0 0 1 1 0 ( Dan , 0) h ( Dan ) = (5 , 8 , 9)             ( Eli , 0) h ( Eve ) = (2 , 8 , 9) 0 1 0 0 0 0 0 1 1 0     ( Fen , 1) h ( Fen ) = (1 , 5 , 6) 1 0 0 0 1 1 0 0 0 1 For ε > 0 . 09 such systems are solvable whp. For ε < 0 . 09 such systems are not solvable whp. For ε > 0 . 23 the system is solvable in O ( m ) time whp.

Retrieval using Linear Systems 1 9 8 7 6 5 4 3 2 2 : Input Hash Values Pick n = (1 + ε ) m and h : U → { 1 , . . . , n } 3 . Yields A ∈ F m × n 2  1  ( Ana , 1) h ( Ana ) = (1 , 3 , 9)  1 0 1 0 0 0 0 0 1   1  0     ( Bea , 1) h ( Bea ) = (2 , 3 , 4) 0 1 1 1 0 0 0 0 0 0 1              1  ( Cal , 0) h ( Cal ) = (3 , 6 , 8) 0 0 1 0 0 1 0 1 0 0       ·   =   0     0 0 0 0 1 0 0 1 1 0 ( Dan , 0) h ( Dan ) = (5 , 8 , 9)       0             ( Eli , 0) h ( Eve ) = (2 , 8 , 9) 0 1 0 0 0 0 0 1 1 0  0          ( Fen , 1) h ( Fen ) = (1 , 5 , 6) 1 0 0 0 1 1 0 0 0 0 1   0 The retrieval data structure R = ( h ,⃗ x ) query ( u ) := ∑ 3 i =1 ⃗ x [ h i ( u )] Memory Requirement: ∼ 1 . 09 m bits (or ∼ 1 . 23 m bits with O ( m ) construction).

Contribution and Previous Work [GOV16] [Por09] m log m Paper k [DP08] [DP08] 3 practical? memory accesses query [Maj+96; BPZ13] Slack 3 0 . 23 � 3 0 . 09 � ≈ e − k ✗ O ( log log m ) ✗ √ O (1) Θ( log log m log m ) ✗ ⟨ this work ⟩ 2 Θ( log m m α ) �

Our Idea: Blocked Linear Systems

Two log-sized Blocks per Line h Eli x 4 3 2 1 h Fen h Dan h Cal h Bea h Ana Hash Values 4 Pick n = (1 + ε ) m , block size ℓ ∈ N and h : U → [ n / ℓ ] × [ n / ℓ ] × F ℓ 2 × F ℓ 2 .

Two log-sized Blocks per Line 1 4 3 2 4 Hash Values Pick n = (1 + ε ) m , block size ℓ ∈ N and h : U → [ n / ℓ ] × [ n / ℓ ] × F ℓ 2 × F ℓ 2 . h ( Ana ) = (1 , 3 , 010 , 110)  0 10 1 10   1  100 110 1 h ( Bea ) = (1 , 4 , 111 , 110)         h ( Cal ) = (1 , 4 , 111 , 010) 111 010 0     · ⃗ x =     h ( Dan ) = (1 , 2 , 010 , 110)  010 110   0          h ( Eli ) = (2 , 4 , 111 , 001) 111 001 0     001 1 h ( Fen ) = (3 , 3 , 111 , 110)

Two log-sized Blocks per Line 1 Theorem 4 3 2 4 Hash Values Pick n = (1 + ε ) m , block size ℓ ∈ N and h : U → [ n / ℓ ] × [ n / ℓ ] × F ℓ 2 × F ℓ 2 . h ( Ana ) = (1 , 3 , 010 , 110)  0 10 1 10   1  100 110 1 h ( Bea ) = (1 , 4 , 111 , 110)         h ( Cal ) = (1 , 4 , 111 , 010) 111 010 0     · ⃗ x =     h ( Dan ) = (1 , 2 , 010 , 110)  010 110   0          h ( Eli ) = (2 , 4 , 111 , 001) 111 001 0     001 1 h ( Fen ) = (3 , 3 , 111 , 110) For a block size of ℓ = 4 log m such systems are solvable whp, provided that ε ≥ 27 log ( m )/ m .

Two log-sized Blocks per Line 1 //O(1) 4 3 2 4 Hash Values Pick n = (1 + ε ) m , block size ℓ ∈ N and h : U → [ n / ℓ ] × [ n / ℓ ] × F ℓ 2 × F ℓ 2 .   0 1 h ( Ana ) = (1 , 3 , 010 , 110)  0 10 1 10   1    1   100 110   1 h ( Bea ) = (1 , 4 , 111 , 110) 0           0   h ( Cal ) = (1 , 4 , 111 , 010) 111 010   0     0   · =       h ( Dan ) = (1 , 2 , 010 , 110)  010 110   1   0        1       h ( Eli ) = (2 , 4 , 111 , 001) 111 001 1 0           1 001 1 h ( Fen ) = (3 , 3 , 111 , 110)   0   0 The retrieval data structure R = ( h ,⃗ x ) query ( u ) := ⟨ p 1 , x [ b 1 ] ⟩ ⊕ ⟨ p 2 , x [ b 2 ] ⟩ where h ( u ) = ( b 1 , b 2 , p 1 , p 2 ) and x [ b ] is the b -th block of x Memory Requirement: (1 + O ( log m m )) m bits.

Efficient Construction

O mC Contribution and Previous Work Word size w . Input partitioned into chunks of size C . w log C mC O [Por09] Paper log m [DP08] w m 5 [Maj+96; BPZ13] k [DP08] memory accesses query Slack t construct 3 0 . 23 O ( m ) O ( m 3 ) ≈ e − k O ( m 3 ) O ( log log m ) √ O (1) Ω( log log m log m ) O ( m ) O ( m 3 ) ⟨ this paper ⟩ 2 Θ( log m m )

Contribution and Previous Work k Word size w . Input partitioned into chunks of size C . [Por09] m Paper log m [DP08] C 5 [DP08][ADR09; GOV16] [Maj+96; BPZ13] t construct memory accesses Slack query 3 0 . 23 O ( m ) O ( m 3 ) O ( mC 2 ≈ e − k + log m w ) O ( m 3 ) O ( log log m ) √ O (1) Ω( log log m log m ) O ( m ) O ( m 3 ) O ( mC 2 ⟨ this paper ⟩ 2 Θ( log m mC ) w log C )