Efficient Nonmyopic Batch Active Search Shali Jiang Gustavo - - PowerPoint PPT Presentation

Efficient Nonmyopic Batch Active Search

1

Gustavo Malkomes Matthew Abbott Benjamin Moseley Roman Garnett Shali Jiang NeurIPS 2018

Many real problems involve searching for valuable items from a large pool of candidates in an iterative fashion

Drug discovery Materials discovery

2

What is active search

3

What is active search

3

What is active search

inference model (gives probabilities)

3

Pr( )

What is active search

select a point

inference model (gives probabilities)

3

Pr( )

What is active search

Oracle

Is it positive?

select a point

inference model (gives probabilities)

3

Pr( )

What is active search

Oracle

Is it positive?

select a point

inference model (gives probabilities) yes/no

3

Pr( )

What is active search

Oracle

Is it positive?

select a point

inference model (gives probabilities) yes/no

3

Pr( )

What is active search

Oracle

Is it positive?

select a point

inference model (gives probabilities) yes/no

3

Pr( )

What is active search

Oracle

Is it positive?

select a point

inference model (gives probabilities) yes/no

3

Pr( )

What is active search

Oracle

Is it positive? Identify as many positives as possible in a given number of queries.

select a point

inference model (gives probabilities) yes/no

3

Pr( )

What is active search

Oracle

Is it positive? Identify as many positives as possible in a given number of queries.

select a point

inference model (gives probabilities)

How?

yes/no

3

Pr( )

What is active search

Oracle

Is it positive? Identify as many positives as possible in a given number of queries. inference model (gives probabilities)

How?

yes/no

3

select a batch

• f points

Pr( )

The most straightforward policy one might think of is greedy: —always choose the points with highest probabilities

4

X∗ = arg max

X

“expected #positives in X”

The most straightforward policy one might think of is greedy: —always choose the points with highest probabilities

4

X∗ = arg max

X

“expected #positives in X”

The most straightforward policy one might think of is greedy: —always choose the points with highest probabilities

It’s efficient but myopic, ignoring what could happen in future

4

X∗ = arg max

X

“expected #positives in X”

The most straightforward policy one might think of is greedy: —always choose the points with highest probabilities

It’s efficient but myopic, ignoring what could happen in future

How can we do better?

4

X∗ = arg max

X

“[expected #positives in X]+

[expected #positives in future conditioned on X]”.

Nonmyopic: consider not only this batch, but also what could happen afterwards!

5

X∗ = arg max

X

“[expected #positives in X]+

[expected #positives in future conditioned on X]”.

Nonmyopic: consider not only this batch, but also what could happen afterwards!

Assume conditional independence after X

5

X∗ = arg max

X

“[expected #positives in X]+

[expected #positives in future conditioned on X]”.

Nonmyopic: consider not only this batch, but also what could happen afterwards!

Assume conditional independence after X

5

Efficient for sequential setting (batch size 1) (Jiang et al. (ICML 2017)).

X∗ = arg max

X

“[expected #positives in X]+

[expected #positives in future conditioned on X]”.

Nonmyopic: consider not only this batch, but also what could happen afterwards!

Combinatorial search in batch setting → two approaches: greedy maximization and sequential simulation Assume conditional independence after X

5

Efficient for sequential setting (batch size 1) (Jiang et al. (ICML 2017)).

Sequential simulation

…

assume the label by a fictional oracle

select 2nd point, conditioned on the assumed label of the 1st select 1st point with a sequential policy

1

6

Sequential simulation

…

assume the label by a fictional oracle

select 2nd point, conditioned on the assumed label of the 1st select 1st point with a sequential policy

1

How?

6

Sequential simulation

…

always negative? sample a label? most likely label? always positive?

assume the label by a fictional oracle

select 2nd point, conditioned on the assumed label of the 1st select 1st point with a sequential policy

1

How?

6

Sequential simulation

…

always negative? sample a label? most likely label? always positive?

Best performing due to encouraging

diversity

assume the label by a fictional oracle

select 2nd point, conditioned on the assumed label of the 1st select 1st point with a sequential policy

1

How?

6

Empirical results

7

Averaged over 1600 experiments (10 drug discovery datasets, 8 batch sizes, and 20 repetitions each)

100 200 300 400 500 100 200 300

number of iterations average number of positives found

greedy batch-ens

exploration exploitation exploitation exploitation

Batching is more efficient, but what are we compromising?

8

Batching is more efficient, but what are we compromising?

T=20

8

Batching is more efficient, but what are we compromising?

b=1 (1 point / iter) * (20 iters)

every point is chosen after

• bserving the outcomes of all

previous points!

T=20

8

Batching is more efficient, but what are we compromising?

b=1 b=5 (1 point / iter) * (20 iters) (5 points / iter) * (4 iters)

every point is chosen after

• bserving the outcomes of all

previous points! points are chosen without observing the outcomes of previously added points in this batch

T=20

8

Batching is more efficient, but what are we compromising?

b=1 b=5

Less adaptive decisions could lead to worse performance!

(1 point / iter) * (20 iters) (5 points / iter) * (4 iters)

every point is chosen after

• bserving the outcomes of all

previous points! points are chosen without observing the outcomes of previously added points in this batch

T=20

8

Batching is more efficient, but what are we compromising?

b=1 b=5

Less adaptive decisions could lead to worse performance!

But how much worse?

(1 point / iter) * (20 iters) (5 points / iter) * (4 iters)

every point is chosen after

• bserving the outcomes of all

previous points! points are chosen without observing the outcomes of previously added points in this batch

T=20

8

Adaptivity gap

We prove that the performance ratio between

• ptimal sequential and batch policies is at least

linear in the batch size!

9

• pt1
• ptb

= Ω ✓ b log T ◆

Adaptivity gap

We prove that the performance ratio between

• ptimal sequential and batch policies is at least

linear in the batch size!

9

• pt1
• ptb

= Ω ✓ b log T ◆

5 25 50 75 100 1 1.2 1.4

batch size average performance ratio

matching empirical results

Adaptivity gap

We prove that the performance ratio between

• ptimal sequential and batch policies is at least

linear in the batch size!

9

• pt1
• ptb

= Ω ✓ b log T ◆

5 25 50 75 100 1 1.2 1.4

batch size average performance ratio

matching empirical results

This insight could help us choose the batch size in cases where we have many options.

Thanks for your attention! Poster: #131

10

Gustavo Malkomes Matthew Abbott Benjamin Moseley Roman Garnett Shali Jiang NeurIPS 2018