Lower Bounds on Non-Adaptive Data Structures Maintaining Sets of - - PowerPoint PPT Presentation

Lower Bounds on Non-Adaptive Data Structures Maintaining Sets

• f Numbers, from Sunflowers

Sivaramakrishnan Natarajan Ramamoorthy, Anup Rao University of Washington

What is the most efficient way to maintain a set of numbers from {1,2,….,n}? — insert new elements — compute the median — compute the predecessors

Universe is [n]

Predecessor(x) = Max {y ≤ x} subject to y ∈ S

S={1,3,5,6,11,13,16}

S={1,3,5,6,11,13,16} Universe is [n]

1 3 5 6 11 13 16

Sorted Array

Time Complexity: insert: n median: 2 predecessor: log n Predecessor(x) = Max {y ≤ x} subject to y ∈ S 7 size

1 3 5 6 11 13 16 Time Complexity: insert: n median: 2 predecessor: log n

Sorted Array Bit Vector

1 1 1 1 1 1 1 Time Complexity: insert: 1 median: n predecessor: n 7 size

S={1,3,5,6,11,13,16} Universe is [n]

Predecessor(x) = Max {y ≤ x} subject to y ∈ S

1 3 5 6 11 13 16 Time Complexity: insert: n median: 2 predecessor: log n

Sorted Array Bit Vector

1 1 1 1 1 1 1 Time Complexity: insert: 1 median: n predecessor: n

1 1 3 3 6 6 6 6 6 11 11 13 13 13 16 5

Predecessor Array

Time Complexity: insert: n median: n predecessor: 1 7 size

S={1,3,5,6,11,13,16} Universe is [n]

Predecessor(x) = Max {y ≤ x} subject to y ∈ S

1 3 6

11 13 16

1 1 1 3 1 6 1

11 0

1

13 0

1

16

1 1 1 3 1 6 1

11

1 1

13

16

2 1 3 6 1 2

11 16

3 3 6

16

#left #right maxL maxR

Operations: insert 5 S={1,3,6,11,13,16}

Binary Search Trees

insertions, median and predecessor in O(log n) time

Binary Search Trees

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

#left #right maxL maxR

Operations: insert 5 S={1,3,6,11,13,16} insertions, median and predecessor in O(log n) time

Binary Search Trees

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

#left #right maxL maxR

Operations: median S={1,3,5,6,11,13,16} insertions, median and predecessor in O(log n) time

Binary Search Trees

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

• utput = 6

#left #right maxL maxR

Operations: median S={1,3,5,6,11,13,16} insertions, median and predecessor in O(log n) time

Binary Search Trees

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

predecessor(15)

#left #right maxL maxR

Operations: S={1,3,5,6,11,13,16} insertions, median and predecessor in O(log n) time

Binary Search Trees

predecessor(15)

#left #right maxL maxR

Operations: S={1,3,5,6,11,13,16}

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

• utput = 13

insertions, median and predecessor in O(log n) time

van Emde Boas Algorithm[77]

S={1,3,11,13,16} insertions, median and predecessor in O(log log n) time Recursion: T(n)=T(√n)+O(1)

(at every level, recurse on Si or S’) 1 1 1 1 1

√n √n √n √n

min=1 max=16 1 1 1 1 1 1 1 1

S1 S2 S3 S4 S’={i:Si≠∅}

min=1 max=3 min=0 max=0 min=3 max=3 min=1 max=4 min=1 max=4

Cell Probe Model [Yao81]

w bits

insert x

Update Algorithm: access subset of cells, modifying some to reflect changes Data structure is a collection of cells that stores the data time = #cells accessed during update

Median

Query Algorithm: access subset of cells and outputs answer w bits 25

Cell Probe Model [Yao81]

Update Algorithm: access subset of cells, modifying some to reflect changes time = #cells accessed during update time = #cells accessed during query Data structure is a collection of cells that stores the data

Non-Adaptive Inserts

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

#left #right maxL maxR

An operation is non-adaptive if locations of cells accessed depends only on the operation and not on contents of the cell

Adaptive Median

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

#left #right maxL maxR

An operation is non-adaptive if locations of cells accessed depends only on the operation and not on contents of the cell

Non-Adaptive Predecessors

#left #right maxL maxR

1 3 5 6

11 13 16

1 1 1 3 1 1 5 6 1

11 0

1

13 0

1

16

1 1 1 3 2 6 1

11

1 1

13

16

2 2 3 6 1 2

11 16

4 3 6

16

An operation is non-adaptive if locations of cells accessed depends only on the operation and not on contents of the cell

S={1,3,11,13,16} insertions, median and predecessor in O(log log n) time

1 1 1 1 1

√n √n √n √n

min=1 max=16 1 1 1 1 1 1 1 1

S1 S2 S3 S4 S’={i:Si≠∅}

min=1 max=3 min=0 max=0 min=3 max=3 min=1 max=4 min=1 max=4

Adaptive Queries and Updates

Advantages of Non-Adaptivity

• Non-adaptive data structures are simpler.
• An implementation perspective: can load all

relevant cells into cache at one go.

Some Known Dynamic Data Structure Lower Bounds

Dynamic connectivity with edge insertions, Parity Sum

[FredmanSaks89] chronogram technique

kth smallest

[PatrascuThorup14] reduction to Parity Sum

Dynamic connectivity with edge insertions + deletions

[PatrascuDemaine04] information transfer technique

2D range counting

[Larsen12] cell sampling + chronogram

2D range counting (amortized + large error)

[WeinsteinYu16] a new communication model

tq ≥ log n log wtu tq ≥ log n log wtu

tq ≥ log2 n log2 wtu tq ≥ log2 n log2 wtu

max{tu, tq} ≥ log n

Adaptive updates and queries

directed graph reachability with non-adaptive queries [BrodyLarsen12]

Non-Adaptive

max{tu, tq} ≥ n/w

non-adaptive insert and predecessor operations

independent [BoningerBrody Kephart17]

non-adaptive predecessor

• peration

this work

adaptive insert and predecessor operation

[A88][BF02][SV04] [PatrascuThorup06]

Predecessor Search

• r

tp ≥ log n log log n + log w

ti ≥ tp · n

1 2(tp+1)

log n

ti + tp ≥ log log n

ti ≥ log n or

tp ≥ min{|S|, w/2} · log n w · log wti

Non-Adaptive Lower Bounds

non-adaptive delete and minimum operations

this work tdel + tmin ≥ log n log log n + log w

non-adaptive insert and median operations

this work

non-adaptive inserts

this work

adaptive insert and median operations

even super constant open

Median

ti + tm ≥ log n log log n + log w

tm ≥ n

1 ti+1

w2 · t2

i

Bit vectors: ti=1, tm=n

Minimum with deletions

Predecessor Arrays: ti=n, tp=1 Binary trees: ti=log n, tp=log n vEB: ti=loglog n, tp=loglog n

Results for Median in Other Models

• Time Space trade offs.

[MunroRaman96,Chan10], [BeameLiewPatrascu15]

• Streaming Algorithms.

[ChakrabartiJayramPatrascu08]

• Data Structures in the Comparison Model.

[BrodalChaudriRadhakrishnan96]

• insert time = t,
• median computation ≥ m/4t, median computation ≤ m/2t

The Key Technical Tool

X10

Sunflower Lemma

[ErdosRado60]

X1 Xm X2 For every sequence X=X1,X2,…,Xn such that |Xi|≤t, if n ≥ t!(p − 1)t+1

Sunflower Lemma

[ErdosRado60]

X1 Xp C X2 For every sequence X=X1,X2,…,Xn such that |Xi|≤t, , then there is a Sunflower with p petals: after renaming, ∃C such that Xi∩Xj=C for distinct i,j ∈ [p] if n ≥ t!(p − 1)t+1

Sunflowers in Complexity

• Monotone Circuit Lower bounds:

[Razborov85] [AlonBoppana87]

• Dynamic data structures with w=1:

[FrandsenMiltersenSkyum93]

• Static data structures with w=1:

[GalMiltersen07] (succinct representations)

First Lower Bound for Median

Theorem 1: Any data structure for Median on [n] with non-adaptive insert and median operations must have

ti + tm ≥ log n log log n + log w w: word size ti: time for inserts tm: time for median

Outline of Proof

• 2. Identify a Sunflower in X with p petals

X1 Xp C Xp/3 X2p/3

• 3. Use C to encode a random subset
• f [p/3,2p/3]
• 1. Let X = X1,X2,….,Xn, where

Xi = {j| j is accessed while inserting i or computing median}

X1 Xp

C

Xp/3 X2p/3

Proof of Theorem 1

Xi = {j| j is accessed while inserting i or computing median}

Proof of Theorem 1

Middle 1/3rd X1 Xp

C

Xp/3 X2p/3

• 1. Let S be a uniformly random subset of

middle 1/3rd elements Xi = {j| j is accessed while inserting i or computing median}

Proof of Theorem 1

X1 Xp

C

Xp/3 X2p/3

• 2. Insert elements of S
• 1. Let S be a uniformly random subset of

middle 1/3rd elements Xi = {j| j is accessed while inserting i or computing median}

Proof of Theorem 1

• 1. Let S be a uniformly random subset of

middle 1/3rd elements

• 2. Insert elements of S

X1 Xp

C

Xp/3 X2p/3 Xi = {j| j is accessed while inserting i or computing median}

Proof of Theorem 1

• 1. Let S be a uniformly random subset of

middle 1/3rd elements Claim: Contents of C encode S

• 2. Insert elements of S

X1 Xp

C

Xp/3 X2p/3 Xi = {j| j is accessed while inserting i or computing median}

Proof of Theorem 1

• 1. Let S be a uniformly random subset of

middle 1/3rd elements Claim: Contents of C encode S

• 2. Insert elements of S

a) If x∈[1,p/3]∪[2p/3+1,p], x can be inserted with access only to C. X1 Xp

C

Xp/3 X2p/3 Xi = {j| j is accessed while inserting i or computing median}

Proof of Theorem 1

• 1. Let S be a uniformly random subset of

middle 1/3rd elements Claim: Contents of C encode S

• 2. Insert elements of S

a) If x∈[1,p/3]∪[2p/3+1,p], x can be inserted with access only to C. X1 Xp

C

Xp/3 X2p/3 Xi = {j| j is accessed while inserting i or computing median}

Proof of Theorem 1

• 1. Let S be a uniformly random subset of

middle 1/3rd elements Claim: Contents of C encode S

• 2. Insert elements of S

a) If x∈[1,p/3]∪[2p/3+1,p], x can be inserted with access only to C. b) Repeatedly insert elements from [1,p/3]∪[2p/3+1,p] and compute the median to recover S. X1 Xp

C

Xp/3 X2p/3 Xi = {j| j is accessed while inserting i or computing median}

Conclusion

• 3. Therefore, ti+tm ≥ p/3w
• 1. If S is a uniformly random subset of [p/3,2p/3],

then H(S) = p/3

• 2. Since contents of C encode S, H(S) ≤ w(ti+tm)

Entropy ≤ length of encoding

First Lower Bound for Median

Theorem 1: Any data structure for Median on [n] with non-adaptive insert and median operations must have

ti + tm ≥ log n log log n + log w w: word size ti: time for inserts tm: time for median

Second Lower Bound for Median

Theorem 2: Any data structure for Median

• n [n] with non-adaptive insertions must have

w: word size ti: time for inserts tm: time for median

tm ≥ n

1 ti+1

w2 · t2

i

Outline of Proof

Step 2: median helps recover the kth smallest Step 3: lower bound for kth smallest via chronogram approach Step 1: Like before, identify a Sunflower among X = X1,X2,….,Xn, where Xi = {j| j is accessed while inserting i} X1 Xp C Xp/3 X2p/3

Reduction to kth smallest

p/3

(input set S is a subset of this interval)

1 p

Claim: After fixing C and |S|, can find kth smallest in time tm

X1 Xp

C

Xp/3 X2p/3

• 1. If x∈[1,p/3] ∪ [2p/3,p], inserting

x requires only access to C.

Xi = {j| j is accessed while inserting i}

Reduction to kth smallest

p/3

(input set S is a subset of this interval)

1 p

Claim: After fixing C and |S|, can find kth smallest in time tm

• 1. If x∈[1,p/3] ∪ [2p/3,p], inserting

x requires only access to C.

• 2. Insert elements ∈[1,p/3] ∪ [2p/

3,p] and compute median

X1 Xp

C

Xp/3 X2p/3 Xi = {j| j is accessed while inserting i}

The Chronogram Approach [FS89] [PT14]

• 1. Applying the “log n” lower bound,

Key insight: epoch analysis ([FS89]) exploiting sunflower structure

• 2. Can be improved to

tm ≥ n

1 2(ti+1)

wti ti · tm ≥ log n log wti

Open Question

Remark 1: Proving a super constant lower bound

• n ti+tm is also open (worst case times assumed).

Is the following true? Remark 2: Proving lower bounds for minimum and maximum under deletions is also open. Any data structure for Median on [n] with w=O(log n) and adaptive insert and median operations must have ti + tm ≥ log log n log log log n