[PPT] - Learning and Optimization for Next Generation Wireless Networks PowerPoint Presentation

SLIDE 1

1 / 30

Learning and Optimization for Next Generation Wireless Networks

Tara Javidi

S. Chiu, A. Lalitha, N. Ronquillo, O. Shayevitz, S. Shubhanshu, Y. Kaspi

SLIDE 2

Learning and Parameter Tuning for Next Generation Networks

⊲

Motivation & Setup Motivation I Motivation II Examles Noisy Search Code to Search Break Experiment Design

2 / 30

SLIDE 3

Learning and Optimization for Next Generation Wireless

Motivation & Setup

⊲ Motivation I

Motivation II Examles Noisy Search Code to Search Break Experiment Design

3 / 30

Next-generation wireless systems are increasingly complex

SLIDE 4

Learning and Optimization for Next Generation Wireless

Motivation & Setup

⊲ Motivation I

Motivation II Examles Noisy Search Code to Search Break Experiment Design

3 / 30

Next-generation wireless systems are increasingly complex
Each layer has an increasingly large number of parameters to

be optimally tuned

SLIDE 5

Learning and Optimization for Next Generation Wireless

Motivation & Setup

⊲ Motivation I

Motivation II Examles Noisy Search Code to Search Break Experiment Design

3 / 30

Next-generation wireless systems are increasingly complex
Each layer has an increasingly large number of parameters to

be optimally tuned

Networks operate at an increasingly diverse settings

SLIDE 6

Learning and Optimization for Next Generation Wireless

Motivation & Setup

⊲ Motivation I

Motivation II Examles Noisy Search Code to Search Break Experiment Design

3 / 30

Next-generation wireless systems are increasingly complex
Each layer has an increasingly large number of parameters to

be optimally tuned

Networks operate at an increasingly diverse settings

–

Performance relies on learning and parameter

ptimization

–

Example: network control’s main task involves iterative enhancements of PHY parameters

SLIDE 7

Learning and Optimization for Next Generation Wireless

Motivation & Setup Motivation I

⊲ Motivation II

Examles Noisy Search Code to Search Break Experiment Design

4 / 30

Unlike in legacy systems the overhead associated with this

learning/optimization can be significant

SLIDE 8

Learning and Optimization for Next Generation Wireless

Motivation & Setup Motivation I

⊲ Motivation II

Examles Noisy Search Code to Search Break Experiment Design

4 / 30

Unlike in legacy systems the overhead associated with this

learning/optimization can be significant

Parameter space is increasingly large and complex

SLIDE 9

Learning and Optimization for Next Generation Wireless

Motivation & Setup Motivation I

⊲ Motivation II

Examles Noisy Search Code to Search Break Experiment Design

4 / 30

Unlike in legacy systems the overhead associated with this

learning/optimization can be significant

Parameter space is increasingly large and complex

–

Ultra Wideband spectrum sensing

–

Ultra narrow beam alignment for mmWave communication

–

Empirical network parameter tuning

SLIDE 10

Learning and Optimization for Next Generation Wireless

Motivation & Setup Motivation I

⊲ Motivation II

Examles Noisy Search Code to Search Break Experiment Design

4 / 30

Unlike in legacy systems the overhead associated with this

learning/optimization can be significant

Parameter space is increasingly large and complex

–

Ultra Wideband spectrum sensing

–

Ultra narrow beam alignment for mmWave communication

–

Empirical network parameter tuning

Our objective is to characterize/minimize the network
verhead associated w learning/optimization

SLIDE 11

Spectrum Sensing and Initial Access

Motivation & Setup

⊲ Examles

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

5 / 30

SLIDE 12

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each

SLIDE 13

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user

SLIDE 14

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user

time 1 . . . τ − 1 τ sample A(1) . . . A(τ − 1)

bservation

Y (1) . . . Y (τ − 1) declaration ˆ W = d(Y τ−1, xτ−1) error 1{ ˆ

W =W }

SLIDE 15

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user

time 1 . . . τ − 1 τ sample A(1) . . . A(τ − 1)

bservation

Y (1) . . . Y (τ − 1) declaration ˆ W = d(Y τ−1, xτ−1) error 1{ ˆ

W =W }

Inspection of a subset results in a signal plus noise

measurement

SLIDE 16

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user
Inspection of a subset results in a signal plus noise

measurement

–

Unit signal associated w the availability of band

–

Sensing noise/unit of spectrum ≈ 0-mean, σ2-var Gaussian

SLIDE 17

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user
Inspection of a subset results in a signal plus noise

measurement Y a = aT (W + Z)

SLIDE 18

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user
Inspection of a subset results in a signal plus noise

measurement Y a = aT (W + Z) a ∈ A, W ∈ {0, 1}

B δ

||W||0 = K Z ∼ N(0, δσ2I)

SLIDE 19

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user
Inspection of a subset results in a signal plus noise

measurement Y a = aT (W + Z)

a, W ∈ {0, 1}

B δ

||W||0 = K N ∼ N(0, Bσ2/δI)

Minimize E{τ} subject to Pe ≤ ǫ

SLIDE 20

Spectrum Sensing: Problem Statement

Motivation & Setup Examles

⊲

Spectrum Sensing Initial Access Noisy Search Code to Search Break Experiment Design

6 / 30

Spectrum with total bandwidth of B is available for

transmission

Primary users have dedicated sub-bands of bandwidth δ each
Subset of subbands inspected sequentially by secondary user
Inspection of a subset results in a signal plus noise

measurement Y a = aT (W + Z)

a, W ∈ {0, 1}

B δ

||W||0 = K N ∼ N(0, Bσ2/δI)

Minimize E{τǫ}

SLIDE 21

Spectrum Sensing: Problem Statement

Motivation & Setup Examles Spectrum Sensing

⊲ Initial Access

Noisy Search Code to Search Break Experiment Design

7 / 30

Directional transmission B ⊂ 2π is available for transmission
Angular resolution of δ ≤ B

SLIDE 22

Spectrum Sensing: Problem Statement

Motivation & Setup Examles Spectrum Sensing

⊲ Initial Access

Noisy Search Code to Search Break Experiment Design

7 / 30

Directional transmission B ⊂ 2π is available for transmission
Angular resolution of δ ≤ B
Subsets of B are used sequentially by transmitter (receiver)

time 1 . . . τ − 1 τ sample A(1) . . . A(τ − 1)

bservation

Y (1) . . . Y (τ − 1) declaration ˆ W = d(Y τ−1, xτ−1) error 1{ ˆ

W =W }

SLIDE 23

Spectrum Sensing: Problem Statement

Motivation & Setup Examles Spectrum Sensing

⊲ Initial Access

Noisy Search Code to Search Break Experiment Design

7 / 30

Directional transmission B ⊂ 2π is available for transmission
Angular resolution of δ ≤ B
Subsets of B are used sequentially by transmitter (receiver)
Inspection of a subset results in a signal plus noise

measurement

SLIDE 24

Spectrum Sensing: Problem Statement

Motivation & Setup Examles Spectrum Sensing

⊲ Initial Access

Noisy Search Code to Search Break Experiment Design

7 / 30

Directional transmission B ⊂ 2π is available for transmission
Angular resolution of δ ≤ B
Subsets of B are used sequentially by transmitter (receiver)
Inspection of a subset results in a signal plus noise

measurement Y a = aT (W + Z)

a, W ∈ {0, 1}

B δ

||W||0 = K Z ∼ N(0, δσ2I)

SLIDE 25

Spectrum Sensing: Problem Statement

Motivation & Setup Examles Spectrum Sensing

⊲ Initial Access

Noisy Search Code to Search Break Experiment Design

7 / 30

Directional transmission B ⊂ 2π is available for transmission
Angular resolution of δ ≤ B
Subsets of B are used sequentially by transmitter (receiver)
Inspection of a subset results in a signal plus noise

measurement Y a = aT (W + Z)

a, W ∈ {0, 1}

B δ

||W||0 = K Z ∼ N(0, δσ2I)

SLIDE 26

Spectrum Sensing: Problem Statement

Motivation & Setup Examles Spectrum Sensing

⊲ Initial Access

Noisy Search Code to Search Break Experiment Design

7 / 30

Directional transmission B ⊂ 2π is available for transmission
Angular resolution of δ ≤ B
Subsets of B are used sequentially by transmitter (receiver)
Inspection of a subset results in a signal plus noise

measurement Y a = aT (W + Z)

a, W ∈ {0, 1}

B δ

||W||0 = K Z ∼ N(0, δσ2I)

Minimize E{τǫ}

SLIDE 27

Measurement-Dependent Noisy Search

Motivation & Setup Examles

⊲ Noisy Search

Problem Setup Questions Analysis I Analysis II Summary Result Code to Search Break Experiment Design

8 / 30

SLIDE 28

Measurement-Dependent Noisy Search

Motivation & Setup Examles Noisy Search

⊲ Problem Setup

Questions Analysis I Analysis II Summary Result Code to Search Break Experiment Design

9 / 30

Uknown parameter: W ∈ {0, 1}

B δ , ||W||0 = 1

Actions A(t) ∈ A ⊂ {0, 1}

B δ chosen sequentially

Y (t) = A(t)(W + Z)

SLIDE 29

Measurement-Dependent Noisy Search

Motivation & Setup Examles Noisy Search

⊲ Problem Setup

Questions Analysis I Analysis II Summary Result Code to Search Break Experiment Design

9 / 30

Uknown parameter: W ∈ {0, 1}

B δ , ||W||0 = 1

Actions A(t) ∈ A ⊂ {0, 1}

B δ chosen sequentially

Y (t) = A(t)(W + Z) = A(t)W + ˆ

Z

–

Observation noise variance increases w |A(t)|

SLIDE 30

Measurement-Dependent Noisy Search

Motivation & Setup Examles Noisy Search

⊲ Problem Setup

Questions Analysis I Analysis II Summary Result Code to Search Break Experiment Design

9 / 30

Uknown parameter: W ∈ {0, 1}

B δ , ||W||0 = 1

Actions A(t) ∈ A ⊂ {0, 1}

B δ chosen sequentially

Y (t) = A(t)(W + Z) = A(t)W + ˆ

Z

–

Observation noise variance increases w |A(t)|

time 1 . . . τ − 1 τ sample A(1) . . . A(τ − 1)

bservation

Y (1) . . . Y (τ − 1) declaration ˆ W = d(Y τ−1, xτ−1) error 1{ ˆ

W =W }

Objective: Find τ, A(0), . . . , A(τ − 1), and d(·) that minimize E [τ] s.t. Pe ≤ ǫ

SLIDE 31

Measurement-Dependent Noisy Search

Motivation & Setup Examles Noisy Search

⊲ Problem Setup

Questions Analysis I Analysis II Summary Result Code to Search Break Experiment Design

9 / 30

Uknown parameter: W ∈ {0, 1}

B δ , ||W||0 = 1

Actions A(t) ∈ A ⊂ {0, 1}

B δ chosen sequentially

Y (t) = A(t)(W + Z) = A(t)W + ˆ

Z

–

Observation noise variance increases w |A(t)|

time 1 . . . τ − 1 τ sample A(1) . . . A(τ − 1)

bservation

Y (1) . . . Y (τ − 1) declaration ˆ W = d(Y τ−1, xτ−1) error 1{ ˆ

W =W }

Objective: Find τ, A(0), . . . , A(τ − 1), and d(·) that minimize E [τ] s.t. Pe ≤ ǫ

Numerical solution via a dynamic programming equation

SLIDE 32

Simpler Questions of General Consequence

Motivation & Setup Examles Noisy Search Problem Setup

⊲ Questions

Analysis I Analysis II Summary Result Code to Search Break Experiment Design

10 / 30

Role of allowable actions set A

SLIDE 33

Simpler Questions of General Consequence

Motivation & Setup Examles Noisy Search Problem Setup

⊲ Questions

Analysis I Analysis II Summary Result Code to Search Break Experiment Design

10 / 30

Role of allowable actions set A

–

Designing A can significantly reduce the overhead

SLIDE 34

Simpler Questions of General Consequence

Motivation & Setup Examles Noisy Search Problem Setup

⊲ Questions

Analysis I Analysis II Summary Result Code to Search Break Experiment Design

10 / 30

Role of allowable actions set A

–

Designing A can significantly reduce the overhead

⊲

Even though noise variance increases w |a| linearly!

SLIDE 35

Simpler Questions of General Consequence

Motivation & Setup Examles Noisy Search Problem Setup

⊲ Questions

Analysis I Analysis II Summary Result Code to Search Break Experiment Design

10 / 30

Role of allowable actions set A

–

Designing A can significantly reduce the overhead

⊲

Even though noise variance increases w |a| linearly!

Selecting A(t) based on past observations (a feedback

scheme) or off-line (non-adaptively)?

SLIDE 36

Simpler Questions of General Consequence

Motivation & Setup Examles Noisy Search Problem Setup

⊲ Questions

Analysis I Analysis II Summary Result Code to Search Break Experiment Design

10 / 30

Role of allowable actions set A

–

Designing A can significantly reduce the overhead

⊲

Even though noise variance increases w |a| linearly!

Selecting A(t) based on past observations (a feedback

scheme) or off-line (non-adaptively)?

–

What is the adaptivity gain?

–

Feedback policies are computationally expensive

SLIDE 37

Role of Measurements

Motivation & Setup Examles Noisy Search Problem Setup Questions

⊲ Analysis I

Analysis II Summary Result Code to Search Break Experiment Design

11 / 30

Role of allowable actions set A

SLIDE 38

Role of Measurements

Motivation & Setup Examles Noisy Search Problem Setup Questions

⊲ Analysis I

Analysis II Summary Result Code to Search Break Experiment Design

11 / 30

Role of allowable actions set A

–

Advantages of group testing

SLIDE 39

Role of Measurements

Motivation & Setup Examles Noisy Search Problem Setup Questions

⊲ Analysis I

Analysis II Summary Result Code to Search Break Experiment Design

11 / 30

Role of allowable actions set A

–

Advantages of group testing

SLIDE 40

Role of Measurements

Motivation & Setup Examles Noisy Search Problem Setup Questions

⊲ Analysis I

Analysis II Summary Result Code to Search Break Experiment Design

11 / 30

Role of allowable actions set A

–

Advantages of group testing

–

If A only singletons (||A(t)|| = 1) ⇒ search time O(B/δ)

–

If A includes intervals, can be O (log(B/δǫ))

SLIDE 41

Role of Measurements

Motivation & Setup Examles Noisy Search Problem Setup Questions

⊲ Analysis I

Analysis II Summary Result Code to Search Break Experiment Design

11 / 30

Role of allowable actions set A

–

Advantages of group testing

–

If A only singletons (||A(t)|| = 1) ⇒ search time O(B/δ)

–

If A includes intervals, can be O (log(B/δǫ)) Observation: If Y a =

X

1{object in a} +Z, Z ∼ N(0, σ2

z), ⇒ E[τ] ≈ log B/δǫ I(X,Y a)

SLIDE 42

Adaptivity Gain

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I

⊲ Analysis II

Summary Result Code to Search Break Experiment Design

12 / 30

Selecting A(t) based on past observations (a feedback

scheme) is computationally expensive

SLIDE 43

Adaptivity Gain

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I

⊲ Analysis II

Summary Result Code to Search Break Experiment Design

12 / 30

Selecting A(t) based on past observations (a feedback

scheme) is computationally expensive

Critical to quantify the Adaptivity (feedback) gain E [τǫ]:

E [τ na

ǫ ] − E [τ ∗ ǫ ]

SLIDE 44

Adaptivity Gain

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I

⊲ Analysis II

Summary Result Code to Search Break Experiment Design

12 / 30

Selecting A(t) based on past observations (a feedback

scheme) is computationally expensive

Critical to quantify the Adaptivity (feedback) gain E [τǫ]:

E [τ na

ǫ ] − E [τ ∗ ǫ ]

Asymptotic analysis when B/δ grows

SLIDE 45

Adaptivity Gain

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I

⊲ Analysis II

Summary Result Code to Search Break Experiment Design

12 / 30

Selecting A(t) based on past observations (a feedback

scheme) is computationally expensive

Critical to quantify the Adaptivity (feedback) gain E [τǫ]:

E [τ na

ǫ ] − E [τ ∗ ǫ ]

Asymptotic analysis when B/δ grows

–

Qualitative difference when B grows versus δ shrinks

SLIDE 46

Adaptivity Gain

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I

⊲ Analysis II

Summary Result Code to Search Break Experiment Design

12 / 30

Selecting A(t) based on past observations (a feedback

scheme) is computationally expensive

Critical to quantify the Adaptivity (feedback) gain E [τǫ]:

E [τ na

ǫ ] − E [τ ∗ ǫ ]

Asymptotic analysis when B/δ grows

–

Qualitative difference when B grows versus δ shrinks

⊲

When B grows overall noise variance grows

⊲

Overall noise is constant even when 1/δ grows

SLIDE 47

Adaptivity Gain

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I

⊲ Analysis II

Summary Result Code to Search Break Experiment Design

12 / 30

Selecting A(t) based on past observations (a feedback

scheme) is computationally expensive

Critical to quantify the Adaptivity (feedback) gain E [τǫ]:

E [τ na

ǫ ] − E [τ ∗ ǫ ]

Asymptotic analysis when B/δ grows

–

Qualitative difference when B grows versus δ shrinks

⊲

When B grows overall noise variance grows

⊲

Overall noise is constant even when 1/δ grows

–

Need for a fairly tight non-asymptotic analysis

SLIDE 48

Our Contributions: Main Take-aways (general K, K = 1)

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I Analysis II

⊲ Summary Result

Code to Search Break Experiment Design

13 / 30

Searching with codebooks with feedback over a stateful

channel

SLIDE 49

Our Contributions: Main Take-aways (general K, K = 1)

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I Analysis II

⊲ Summary Result

Code to Search Break Experiment Design

13 / 30

Searching with codebooks with feedback over a stateful

channel

Zn Yn (1) (r) (2)

SLIDE 50

Our Contributions: Main Take-aways (general K, K = 1)

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I Analysis II

⊲ Summary Result

Code to Search Break Experiment Design

13 / 30

Searching with codebooks with feedback over a stateful

channel (K = 1)

–

Reduces the non-adaptive case to known IT problems

–

Adaptive strategy as a variant of feedback code

SLIDE 51

Our Contributions: Main Take-aways (general K, K = 1)

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I Analysis II

⊲ Summary Result

Code to Search Break Experiment Design

13 / 30

Searching with codebooks with feedback over a stateful

channel (K = 1)

–

Reduces the non-adaptive case to known IT problems

–

Adaptive strategy as a variant of feedback code

Non-asymptotic achievability analysis for an adaptive scheme

–

Sorted Posterior Matching (SortPM) search strategy

SLIDE 52

Our Contributions: Main Take-aways (general K, K = 1)

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I Analysis II

⊲ Summary Result

Code to Search Break Experiment Design

13 / 30

Searching with codebooks with feedback over a stateful

channel (K = 1)

–

Reduces the non-adaptive case to known IT problems

–

Adaptive strategy as a variant of feedback code

Non-asymptotic achievability analysis for an adaptive scheme

–

Sorted Posterior Matching (SortPM) search strategy

Characterize daptivity gain with two distinct asymptotic

regimes B/δ → ∞

SLIDE 53

Our Contributions: Main Take-aways (general K, K = 1)

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I Analysis II

⊲ Summary Result

Code to Search Break Experiment Design

13 / 30

Searching with codebooks with feedback over a stateful

channel (K = 1)

–

Reduces the non-adaptive case to known IT problems

–

Adaptive strategy as a variant of feedback code

Non-asymptotic achievability analysis for an adaptive scheme

–

Sorted Posterior Matching (SortPM) search strategy

Characterize daptivity gain with two distinct asymptotic

regimes B/δ → ∞

–

Fixed search interval and increasing resolution (initial access)

–

Fixed resolution and increasing search (primary user detection)

SLIDE 54

Our Contributions: Main Take-aways (general K, K = 1)

Motivation & Setup Examles Noisy Search Problem Setup Questions Analysis I Analysis II

⊲ Summary Result

Code to Search Break Experiment Design

13 / 30

Searching with codebooks with feedback over a stateful

channel (K = 1)

–

Reduces the non-adaptive case to known IT problems

–

Adaptive strategy as a variant of feedback code

Non-asymptotic achievability analysis for an adaptive scheme

–

Sorted Posterior Matching (SortPM) search strategy

Characterize daptivity gain with two distinct asymptotic

regimes B/δ → ∞

–

Fixed search interval and increasing resolution (initial access)

–

Fixed resolution and increasing search (primary user detection)

SLIDE 55

Analysis

Motivation & Setup Examles Noisy Search

⊲ Code to Search

Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

14 / 30

SLIDE 56

Non-asymptotic Converse for Non-adaptive Search:

Motivation & Setup Examles Noisy Search Code to Search

⊲ Non-adaptive

Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

15 / 30

Searching via coding over a stateful channel

SLIDE 57

Non-asymptotic Converse for Non-adaptive Search:

Motivation & Setup Examles Noisy Search Code to Search

⊲ Non-adaptive

Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

15 / 30

Searching via coding over a stateful channel

Zn Yn (1) (r) (2)

SLIDE 58

Non-asymptotic Converse for Non-adaptive Search:

Motivation & Setup Examles Noisy Search Code to Search

⊲ Non-adaptive

Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

15 / 30

Searching via coding over a stateful channel

Zn Yn (1) (r) (2)

–

Reduces non-adaptive case to known IT problem:

Y = Xq + Zq, Xq ∼ Ber(q), Zq ∼ N(0, qB

δ σ2)

SLIDE 59

Non-asymptotic Converse for Non-adaptive Search:

Motivation & Setup Examles Noisy Search Code to Search

⊲ Non-adaptive

Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

15 / 30

Searching via coding over a stateful channel

Zn Yn (1) (r) (2)

–

Reduces non-adaptive case to known IT problem:

Y = Xq + Zq, Xq ∼ Ber(q), Zq ∼ N(0, qB

δ σ2)

E[τ NA

ǫ

] ≥ (1 − ǫ) log B

δ − h(ǫ)

CBPSK(q, σ

qB/δ)

SLIDE 60

Non-asymptotic Converse for Non-adaptive Search:

Motivation & Setup Examles Noisy Search Code to Search

⊲ Non-adaptive

Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

15 / 30

Searching via coding over a stateful channel

Zn Yn (1) (r) (2)

–

Reduces non-adaptive case to known IT problem:

Y = Xq + Zq, Xq ∼ Ber(q), Zq ∼ N(0, qB

δ σ2)

E[τ NA

ǫ

] ≥ (1 − ǫ) log B

δ − h(ǫ)

CBPSK(q∗, σ

q∗B/δ)

SLIDE 61

Non-asymptotic Converse for Non-adaptive Search:

Motivation & Setup Examles Noisy Search Code to Search

⊲ Non-adaptive

Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

15 / 30

Searching via coding over a stateful channel

–

Reduces non-adaptive case to known IT problem:

Y = Xq + Zq, Xq ∼ Ber(q), Zq ∼ N(0, qB

δ σ2)

E[τ NA

ǫ

] ≥ (1 − ǫ) log B

δ − h(ǫ)

CBPSK(q∗, σ

q∗B/δ)

SLIDE 62

Non-asymptotic Converse for Non-adaptive Search:

Motivation & Setup Examles Noisy Search Code to Search

⊲ Non-adaptive

Search Strategies Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

15 / 30

Searching via coding over a stateful channel

–

Reduces non-adaptive case to known IT problem:

Y = Xq + Zq, Xq ∼ Ber(q), Zq ∼ N(0, qB

δ σ2)

E[τ NA

ǫ

] ≥ (1 − ǫ) log B

δ − h(ǫ)

CBPSK(q∗, σ

q∗B/δ)

SLIDE 63

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

SLIDE 64

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Non-adaptive Strategy:

SLIDE 65

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Non-adaptive Strategy: Fix the number of samples τ = T

SLIDE 66

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Non-adaptive Strategy: Fix the number of samples τ = T

select T to be such that E{Pe} ≤ ǫ

SLIDE 67

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Non-adaptive Strategy: Fix the number of samples τ = T

select T to be such that E{Pe} ≤ ǫ
for all t ≤ T query random set a such that |a| = q∗B/δ
ptimized

SLIDE 68

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Sorted Posterior Matching (sortPM) Strategy: Consider prior ρ(t) := (P{W = ei|A(0 : t − 1), Y (0, t − 1)})

declares i as the target, if ρi(t) ≥ 1 − ǫ, i ∈ Ω
therwise, queries the bins left of the median of the sorted

prior

–

bserve (noisy) Y

–

update the prior (posterior) via the Bayes’ rule

SLIDE 69

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Sorted Posterior Matching (sortPM) Strategy: Consider prior ρ(t) := (P{W = ei|A(0 : t − 1), Y (0, t − 1)})

declares i as the target, if ρi(t) ≥ 1 − ǫ, i ∈ Ω
therwise, queries the bins left of the median of the sorted

prior

–

bserve (noisy) Y

–

update the prior (posterior) via the Bayes’ rule

SLIDE 70

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Sorted Posterior Matching (sortPM) Strategy: Consider prior ρ(t) := (P{W = ei|A(0 : t − 1), Y (0, t − 1)})

declares i as the target, if ρi(t) ≥ 1 − ǫ, i ∈ Ω
therwise, queries the bins left of the median of the sorted

prior

–

bserve (noisy) Y

–

update the prior (posterior) via the Bayes’ rule

SLIDE 71

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Sorted Posterior Matching (sortPM) Strategy: Consider prior ρ(t) := (P{W = ei|A(0 : t − 1), Y (0, t − 1)})

declares i as the target, if ρi(t) ≥ 1 − ǫ, i ∈ Ω
therwise, queries the bins left of the median of the sorted

prior

–

bserve (noisy) Y

–

update the prior (posterior) via the Bayes’ rule

SLIDE 72

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Sorted Posterior Matching (sortPM) Strategy: Consider prior ρ(t) := (P{W = ei|A(0 : t − 1), Y (0, t − 1)})

declares i as the target, if ρi(t) ≥ 1 − ǫ, i ∈ Ω
therwise, queries the bins left of the median of the sorted

prior

–

bserve (noisy) Y

–

update the prior (posterior) via the Bayes’ rule

SLIDE 73

Non-adaptive and Adaptive Search Strategies

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive

⊲ Search Strategies

Upper Bound Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

16 / 30

Sorted Posterior Matching (sortPM) Strategy: Consider prior ρ(t) := (P{W = ei|A(0 : t − 1), Y (0, t − 1)})

declares i as the target, if ρi(t) ≥ 1 − ǫ, i ∈ Ω
therwise, queries the bins left of the median of the sorted

prior

–

bserve (noisy) Y

–

update the prior (posterior) via the Bayes’ rule

SLIDE 74

SortPM: Upper Bound

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies

⊲ Upper Bound

Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

17 / 30

Theorem. [Lalitha, Ronquillo and J. 17] Under SortPM, we have

E[τSP M] ≤ min

α

log B/δǫ + max{log log B/δ, log log 1

ǫ }

1 − h(Q((σ2αB/δ)−1/2)) + K(α).

where h(p) = p log 1 p + (1 − p) log 1 1 − p, K(·) is non-increasing function

SLIDE 75

SortPM: Upper Bound

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies

⊲ Upper Bound

Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

17 / 30

Theorem. [Lalitha, Ronquillo and J. 17] Under SortPM, we have

E[τSP M] ≤ min

α

log B/δǫ + max{log log B/δ, log log 1

ǫ }

1 − h(Q((σ2αB/δ)−1/2)) + K(α).

where h(p) = p log 1 p + (1 − p) log 1 1 − p, K(·) is non-increasing function

Analysis is based on a Lyapunov drift

SLIDE 76

SortPM: Upper Bound

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies

⊲ Upper Bound

Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

17 / 30

Corollary. [Lalitha, Ronqullio and J. 17] Relying on hard-detected
utput symbols, the asymptotic adaptivity gain for B/δ → ∞ is:

lim

δ→0

τ NA

pt − E[τ A
pt]

log B

δ

= 1 CBPSK(q∗, Bσ2) − 1. lim

B→∞

τ NA

pt − E[τ A
pt]

B δ log B δ

≥ σ2δ log e.

SLIDE 77

SortPM: Upper Bound

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies

⊲ Upper Bound

Prior Work Generalizations I Generalizations II Generalization III Break Experiment Design

17 / 30

Corollary. [Lalitha, Ronqullio and J. 17] Relying on hard-detected
utput symbols, the asymptotic adaptivity gain for B/δ → ∞ is:

lim

δ→0

τ NA

pt − E[τ A
pt]

log B

δ

= 1 CBPSK(q∗, Bσ2) − 1. lim

B→∞

τ NA

pt − E[τ A
pt]

B δ log B δ

≥ σ2δ log e.

SLIDE 78

Prior Work: Measurement Independent Noise

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound

⊲ Prior Work

Generalizations I Generalizations II Generalization III Break Experiment Design

18 / 30

Generalized binary search [Burnashev and Zigangirov ’74]
Channel coding over DMC with feedback [Burnashev ’75],

[Yamamato and Itoh ’79], ... [Naghshvar, Wigger and J ’13]

Posterior matching [Shayevitz and Feder ’11]
Bisection search with noisy responses [Horstein ’63],

[Waeber, Frazier, Henderson ’13]

SLIDE 79

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

SLIDE 80

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

101 102 8 10 12 14 16 18 20 22 24

SLIDE 81

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

101 102 8 10 12 14 16 18 20 22 24

SLIDE 82

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

101 102 8 10 12 14 16 18 20 22 24

SLIDE 83

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

Fixed (hierarchical) beam patterns

SLIDE 84

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

Fixed (hierarchical) beam patterns

SLIDE 85

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

Fixed (hierarchical) beam patterns

SLIDE 86

Generalizations

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work

⊲ Generalizations I

Generalizations II Generalization III Break Experiment Design

19 / 30

General noise model: Y (t) = A(t)W + ˆ

Z, ˆ Z = f(Z, A(t))

Fixed (hierarchical) beam patterns

SLIDE 87

Generalizations and On-going Work

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I

⊲ Generalizations II

Generalization III Break Experiment Design

20 / 30

Search for multiple target (K > 1)
Dynamic case: W(t)
Beyond Gaussian

SLIDE 88

Generalizations and On-going Work

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I

⊲ Generalizations II

Generalization III Break Experiment Design

20 / 30

Search for multiple target (K > 1)
Noisy sequential group testing [Atia and Saligrama ’12];

Mapped to an OR MAC [Kaspi, Shayevitz, J ’15]

Dynamic case: W(t)
Beyond Gaussian

SLIDE 89

Generalizations and On-going Work

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I

⊲ Generalizations II

Generalization III Break Experiment Design

20 / 30

Search for multiple target (K > 1)
Noisy sequential group testing [Atia and Saligrama ’12];

Mapped to an OR MAC [Kaspi, Shayevitz, J ’15]

Factor of

1 K in rate, where K bounds (is) the number of

targets

Dynamic case: W(t)
Beyond Gaussian

SLIDE 90

Generalizations and On-going Work

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I

⊲ Generalizations II

Generalization III Break Experiment Design

20 / 30

Search for multiple target (K > 1)
Noisy sequential group testing [Atia and Saligrama ’12];

Mapped to an OR MAC [Kaspi, Shayevitz, J ’15]

Factor of

1 K in rate, where K bounds (is) the number of

targets ‡ Case of an adder channel

Dynamic case: W(t)
Beyond Gaussian

SLIDE 91

Generalizations and On-going Work

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I

⊲ Generalizations II

Generalization III Break Experiment Design

20 / 30

Search for multiple target (K > 1)
Noisy sequential group testing [Atia and Saligrama ’12];

Mapped to an OR MAC [Kaspi, Shayevitz, J ’15]

Factor of

1 K in rate, where K bounds (is) the number of

targets ‡ Case of an adder channel

Dynamic case: W(t)
Results generalizes to unknown but constant speed (cut

rate by half)

Beyond Gaussian

SLIDE 92

Generalizations and On-going Work

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I

⊲ Generalizations II

Generalization III Break Experiment Design

20 / 30

Search for multiple target (K > 1)
Noisy sequential group testing [Atia and Saligrama ’12];

Mapped to an OR MAC [Kaspi, Shayevitz, J ’15]

Factor of

1 K in rate, where K bounds (is) the number of

targets ‡ Case of an adder channel

Dynamic case: W(t)
Results generalizes to unknown but constant speed (cut

rate by half)

Beyond Gaussian
Similar results for the binary symmetric noise (hard

decoding) [Kaspi, Shayevitz, J ’14]

SLIDE 93

Empirical Network Parameter tuning

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I Generalizations II

⊲ Generalization III

Break Experiment Design

21 / 30

Network performance function of network parameter f : X → R. Assumptions:

X is the set of network parameters and protocols
f(x) is the network performance; f(x1) and f(x2)

”correlated”

f observed w noise: y = f(x) + η(x), η non-presistent noise

Goal: Design a sequential strategy of selecting n query points x1, . . . , xn to identify a global optimizer of f.

Performance measures:

–

Simple regret: Sn = f(x∗) − f(x∗

n)

–

Cumulative regret: Rn = n

t=1 f(x∗) − f(xt)

SLIDE 94

Empirical Network Parameter tuning

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I Generalizations II

⊲ Generalization III

Break Experiment Design

21 / 30

Network performance function of network parameter f : X → R. Assumptions:

X is the set of network parameters and protocols
f(x) is the network performance; f(x1) and f(x2)

”correlated”

f observed w noise: y = f(x) + η(x), η non-presistent noise

Goal: Design a sequential strategy of selecting n query points x1, . . . , xn to identify a global optimizer of f.

Performance measures:

–

Simple regret: Sn = f(x∗) − f(x∗

n)

–

Cumulative regret: Rn = n

t=1 f(x∗) − f(xt)

SLIDE 95

Empirical Network Parameter tuning

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I Generalizations II

⊲ Generalization III

Break Experiment Design

21 / 30

Network performance function of network parameter f : X → R. Assumptions:

X is the set of network parameters and protocols
f(x) is the network performance; f(x1) and f(x2)

”correlated”

f observed w noise: y = f(x) + η(x), η non-presistent noise

Goal: Design a sequential strategy of selecting n query points x1, . . . , xn to identify a global optimizer of f.

Performance measures:

–

Simple regret: Sn = f(x∗) − f(x∗

n)

–

Cumulative regret: Rn = n

t=1 f(x∗) − f(xt)

SLIDE 96

Empirical Network Parameter tuning

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I Generalizations II

⊲ Generalization III

Break Experiment Design

21 / 30

Network performance function of network parameter f : X → R. Assumptions:

X is the set of network parameters and protocols
f(x) is the network performance; f(x1) and f(x2)

”correlated”

f observed w noise: y = f(x) + η(x), η non-presistent noise

Goal: Design a sequential strategy of selecting n query points x1, . . . , xn to identify a global optimizer of f.

Performance measures:

–

Simple regret: Sn = f(x∗) − f(x∗

n)

–

Cumulative regret: Rn = n

t=1 f(x∗) − f(xt)

SLIDE 97

Empirical Network Parameter tuning

Motivation & Setup Examles Noisy Search Code to Search Non-adaptive Search Strategies Upper Bound Prior Work Generalizations I Generalizations II

⊲ Generalization III

Break Experiment Design

21 / 30

Network performance function of network parameter f : X → R. Assumptions:

X is the set of network parameters and protocols
f(x) is the network performance; f(x1) and f(x2)

”correlated”

f observed w noise: y = f(x) + η(x), η non-presistent noise

Goal: Design a sequential strategy of selecting n query points x1, . . . , xn to identify a global optimizer of f.

Performance measures:

–

Simple regret: Sn = f(x∗) − f(x∗

n)

–

Cumulative regret: Rn = n

t=1 f(x∗) − f(xt) [bandit]

SLIDE 98

Questions?

Motivation & Setup Examles Noisy Search Code to Search

⊲ Break

Experiment Design

22 / 30

SLIDE 99

Experiment Design: Single-shot

Motivation & Setup Examles Noisy Search Code to Search Break

⊲

Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

23 / 30

SLIDE 100

Design of Experiments [Blackwell ’51]

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design

⊲

Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

24 / 30

M mutually exclusive hypotheses: Hi ⇔ {θ = i},

i = 1, 2, . . . , M

Prior ρ(0) = [ρ1(0), . . . , ρM(0)], ρi(0) = P(θ = i)

SLIDE 101

Design of Experiments [Blackwell ’51]

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design

⊲

Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

24 / 30

M mutually exclusive hypotheses: Hi ⇔ {θ = i},

i = 1, 2, . . . , M

Prior ρ(0) = [ρ1(0), . . . , ρM(0)], ρi(0) = P(θ = i)
Experiments A are available

SLIDE 102

Design of Experiments [Blackwell ’51]

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design

⊲

Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

24 / 30

M mutually exclusive hypotheses: Hi ⇔ {θ = i},

i = 1, 2, . . . , M

Prior ρ(0) = [ρ1(0), . . . , ρM(0)], ρi(0) = P(θ = i)
Experiments A are available
Z|{θ=i,A=a} ∼ qa

i (·): observation density given a ∈ A and Hi

SLIDE 103

Design of Experiments [Blackwell ’51]

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design

⊲

Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

24 / 30

M mutually exclusive hypotheses: Hi ⇔ {θ = i},

i = 1, 2, . . . , M

Prior ρ(0) = [ρ1(0), . . . , ρM(0)], ρi(0) = P(θ = i)
Experiments A are available
Z|{θ=i,A=a} ∼ qa

i (·): observation density given a ∈ A and Hi

Objective: What is the best experiment A = a to identify θ?

SLIDE 104

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

SLIDE 105

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

SLIDE 106

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be?

SLIDE 107

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be? Compare experiment a with a′?

SLIDE 108

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be? Compare experiment a with a′?

–

Stochastically degraded case [Blackwell ’53], [Stein ’53]

SLIDE 109

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be? Compare experiment a with a′?

–

Stochastically degraded case [Blackwell ’53], [Stein ’53]

Given experiment a:

SLIDE 110

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be? Compare experiment a with a′?

–

Stochastically degraded case [Blackwell ’53], [Stein ’53]

Given experiment a:

–

True hypothesis θ = i with probability ρi

SLIDE 111

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be? Compare experiment a with a′?

–

Stochastically degraded case [Blackwell ’53], [Stein ’53]

Given experiment a:

–

True hypothesis θ = i with probability ρi

–

Output distribution Za ∼ M

i=1 ρiqa i (·)

SLIDE 112

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be? Compare experiment a with a′?

–

Stochastically degraded case [Blackwell ’53], [Stein ’53]

Given experiment a:

–

True hypothesis θ = i with probability ρi

–

Output distribution Za ∼ M

i=1 ρiqa i (·)

–

Posterior upon observation θ|Za ∼ Φa(ρ, Za) − − −

Bayes operator

SLIDE 113

Learning from a Single Experiment

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design

⊲

Intuitive Overview Heuristic Approaches Notations Mutual Information EJS Achievability

25 / 30

Consider a single experiment a ∈ A

–

Prior θ ∼ ρ

–

Noisy observations subject to {qa

i (·)}i,a

What should a be? Compare experiment a with a′?

–

Stochastically degraded case [Blackwell ’53], [Stein ’53]

Given experiment a:

–

True hypothesis θ = i with probability ρi

–

Output distribution Za ∼ M

i=1 ρiqa i (·)

–

Posterior upon observation θ|Za ∼ Φa(ρ, Za) − − −

Bayes operator

–

How does Φa(·, ·) compare with Φa′(·, ·)

SLIDE 114

Heuristic Methods

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview

⊲

Heuristic Approaches Notations Mutual Information EJS Achievability

26 / 30

Divergence-based Selection

Define a “symmetrized divergence” among qa

1, qa 2, . . . , qa M

SLIDE 115

Heuristic Methods

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview

⊲

Heuristic Approaches Notations Mutual Information EJS Achievability

26 / 30

Divergence-based Selection

Define a “symmetrized divergence” among qa

1, qa 2, . . . , qa M

Best action must maximize the divergence

SLIDE 116

Heuristic Methods

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview

⊲

Heuristic Approaches Notations Mutual Information EJS Achievability

26 / 30

Divergence-based Selection

Define a “symmetrized divergence” among qa

1, qa 2, . . . , qa M

Best action must maximize the divergence

–

maximize discrimination among H1, H2, . . . , HM

SLIDE 117

Heuristic Methods

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview

⊲

Heuristic Approaches Notations Mutual Information EJS Achievability

26 / 30

Divergence-based Selection

Define a “symmetrized divergence” among qa

1, qa 2, . . . , qa M

Best action must maximize the divergence

–

maximize discrimination among H1, H2, . . . , HM Information Utility Heuristics:

Measure of uncertainty V [DeGroot 1962]
Information utility associated with V

IU(a, ρ, V ) = V (ρ) − E[V (Φa(ρ, Z))]

SLIDE 118

Heuristic Methods

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview

⊲

Heuristic Approaches Notations Mutual Information EJS Achievability

26 / 30

Divergence-based Selection

Define a “symmetrized divergence” among qa

1, qa 2, . . . , qa M

Best action must maximize the divergence

–

maximize discrimination among H1, H2, . . . , HM Information Utility Heuristics:

Measure of uncertainty V [DeGroot 1962]
Information utility associated with V

IU(a, ρ, V ) = V (ρ) − E[V (Φa(ρ, Z))] − − −

Bayes operator

SLIDE 119

Heuristic Methods

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview

⊲

Heuristic Approaches Notations Mutual Information EJS Achievability

26 / 30

Divergence-based Selection

Define a “symmetrized divergence” among qa

1, qa 2, . . . , qa M

Best action must maximize the divergence

–

maximize discrimination among H1, H2, . . . , HM Information Utility Heuristics:

Measure of uncertainty V [DeGroot 1962]
Information utility associated with V

IU(a, ρ, V ) = V (ρ) − E[V (Φa(ρ, Z))] − − −

Bayes operator

Most informative action arg maxa IU(a, ρ, V )

SLIDE 120

Heuristic Methods

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview

⊲

Heuristic Approaches Notations Mutual Information EJS Achievability

26 / 30

Divergence-based Selection

Define a “symmetrized divergence” among qa

1, qa 2, . . . , qa M

Best action must maximize the divergence

–

maximize discrimination among H1, H2, . . . , HM Information Utility Heuristics:

Measure of uncertainty V [DeGroot 1962]
Information utility associated with V

IU(a, ρ, V ) = V (ρ) − E[V (Φa(ρ, Z))] − − −

Bayes operator

Most informative action arg maxa IU(a, ρ, V )

Noisy search reduces to maximizing the IU(a, ρ, V ∗)

SLIDE 121

Recall

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches

⊲ Notations

Mutual Information EJS Achievability

27 / 30

The Entropy of p(·) on space Z:

H(p) =

Z

p(z) log 1 p(z)

The Kullback-Leibler (KL) divergence between p(·) and q(·):

D(p||q) =

Z

p(z) log p(z) q(z)

SLIDE 122

Mutual Information as Information Utility

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations

⊲

Mutual Information EJS Achievability

28 / 30

A widely used heuristic πI(ρ) = arg max

a

I(θ; Za), where Za ∼ qa

ρ = M

i=1

ρiqa

i

[Chaloner Verdinelli 1995], [Lindley 1956], [MacKay 1992], [Paninski 2005], [Branson 2010], [Butko Movellan 2009], [Fleuret 2004], [Williams et al. 2007]

SLIDE 123

Mutual Information as Information Utility

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations

⊲

Mutual Information EJS Achievability

28 / 30

A widely used heuristic πI(ρ) = arg max

a

I(θ; Za), where Za ∼ qa

ρ = M

i=1

ρiqa

i

I(θ; Za) = H(ρ) − E(H(Φa(ρ, Za))) = IU(a, ρ, H)

SLIDE 124

Mutual Information as Information Utility

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations

⊲

Mutual Information EJS Achievability

28 / 30

A widely used heuristic πI(ρ) = arg max

a

I(θ; Za), where Za ∼ qa

ρ = M

i=1

ρiqa

i

I(θ; Za) = H(ρ) − E(H(Φa(ρ, Za))) = IU(a, ρ, H) Also I(θ; Za) = M

i=1 ρiD(qa i ||qa ρ)

SLIDE 125

Mutual Information as Information Utility

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations

⊲

Mutual Information EJS Achievability

28 / 30

A widely used heuristic πI(ρ) = arg max

a

I(θ; Za), where Za ∼ qa

ρ = M

i=1

ρiqa

i

I(θ; Za) = H(ρ) − E(H(Φa(ρ, Za))) = IU(a, ρ, H) Also I(θ; Za) = M

i=1 ρiD(qa i ||qa ρ)

Jensen-Shannon divergence [Lin 1991] Generalizing L divergence: DL(f, g) = 1

2D(f|| f+g 2 ) + 1 2D(g|| f+g 2 )

SLIDE 126

Mutual Information as Information Utility

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations

⊲

Mutual Information EJS Achievability

28 / 30

A widely used heuristic πI(ρ) = arg max

a

I(θ; Za), where Za ∼ qa

ρ = M

i=1

ρiqa

i

I(θ; Za) = H(ρ) − E(H(Φa(ρ, Za))) = IU(a, ρ, H) Also I(θ; Za) = M

i=1 ρiD(qa i ||qa ρ)

SLIDE 127

Mutual Information as Information Utility

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations

⊲

Mutual Information EJS Achievability

28 / 30

A widely used heuristic πI(ρ) = arg max

a

I(θ; Za), where Za ∼ qa

ρ = M

i=1

ρiqa

i

I(θ; Za) = H(ρ) − E(H(Φa(ρ, Za))) = IU(a, ρ, H) Also I(θ; Za) = M

i=1 ρiD(qa i ||qa ρ)

As ρi → 1, D(qa

i ||qa ρ) → D(qa i ||qa i ) = 0 for any experiment a

SLIDE 128

A New Symmetrized Divergence

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information

⊲ EJS

Achievability

29 / 30

SLIDE 129

A New Symmetrized Divergence

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information

⊲ EJS

Achievability

29 / 30

Extrinsic Jensen-Shannon Divergence [Naghshvar, J. ISIT’12]

SLIDE 130

A New Symmetrized Divergence

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information

⊲ EJS

Achievability

29 / 30

Extrinsic Jensen-Shannon Divergence [Naghshvar, J. ISIT’12]

The Extrinsic Jensen-Shannon (EJS) divergence among densities q1, q2, . . . , qM with respect to ρ = [ρ1, ρ2, . . . , ρM] is defined as EJS(ρ; q1, q2, . . . , qM) = M

i=1 ρiD(qi|| k=i ρk 1−ρi qk).

SLIDE 131

A New Symmetrized Divergence

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information

⊲ EJS

Achievability

29 / 30

Extrinsic Jensen-Shannon Divergence [Naghshvar, J. ISIT’12]

The Extrinsic Jensen-Shannon (EJS) divergence among densities q1, q2, . . . , qM with respect to ρ = [ρ1, ρ2, . . . , ρM] is defined as EJS(ρ; q1, q2, . . . , qM) = M

i=1 ρiD(qi|| k=i ρk 1−ρi qk).

Bayesian generalization of J-divergence [Jefferys 73]

DJ(f, g) = 1 2 D(f||g) + 1 2D(g||f)

SLIDE 132

A New Symmetrized Divergence

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information

⊲ EJS

Achievability

29 / 30

Extrinsic Jensen-Shannon Divergence [Naghshvar, J. ISIT’12]

The Extrinsic Jensen-Shannon (EJS) divergence among densities q1, q2, . . . , qM with respect to ρ = [ρ1, ρ2, . . . , ρM] is defined as EJS(ρ; q1, q2, . . . , qM) = M

i=1 ρiD(qi|| k=i ρk 1−ρi qk).

Bayesian generalization of J-divergence [Jefferys 73]

DJ(f, g) = 1 2 D(f||g) + 1 2D(g||f)

Proposition EJS is the information utility associated with the average likelihood function U(ρ) = M

i=1 ρi log 1−ρi ρi , i.e.

EJS(ρ; qa

1, . . . , qa M) = IU(a, ρ, U)

SLIDE 133

An Upper Bound on Expected Number of Searches

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information EJS

⊲ Achievability

30 / 30

Theorem (Naghshvar et. al. 13). Suppose there is C > 0 s.t. when a is selected according to SortPM and |a| ≤ αB/δ, for all ρ, EJS(ρ, a) ≥ C. Then

E[τ ∗] ≤ E[τSortP M] ≤ log M + max{log log M, log 1

δ } + 4∆

C + K(α).

SLIDE 134

An Upper Bound on Expected Number of Searches

Motivation & Setup Examles Noisy Search Code to Search Break Experiment Design Experiment Design Intuitive Overview Heuristic Approaches Notations Mutual Information EJS

⊲ Achievability

30 / 30

Theorem (Naghshvar et. al. 13). Suppose there is C > 0 s.t. when a is selected according to SortPM and |a| ≤ αB/δ, for all ρ, EJS(ρ, a) ≥ C. Then

E[τ ∗] ≤ E[τSortP M] ≤ log M + max{log log M, log 1

δ } + 4∆

C + K(α).

Lemma. Fix α ∈ (0, 1). Using hard-decoded observation

sequence ⇒ C(α) = 1 − h

Q
σ2αB/δ