From heuristic to optimal models in naturalistic visual search - - PowerPoint PPT Presentation

▶
from heuristic to optimal models in naturalistic visual
SMART_READER_LITE
LIVE PREVIEW

From heuristic to optimal models in naturalistic visual search - - PowerPoint PPT Presentation

Jan 07, 2024 142 likes •508 views

From heuristic to optimal models in naturalistic visual search Angela Radulescu 1,2 *, Bas van Opheusden 1,2 *, Fred Callaway 2 , Thomas Griffiths 2 & James Hillis 1 1 2 Bridging AI and Cognitive Science workshop, ICLR April 24th, 2020

slide-1
SLIDE 1

From heuristic to optimal models in naturalistic visual search

  • 1

Angela Radulescu1,2*, Bas van Opheusden1,2*, Fred Callaway2, Thomas Griffiths2 & James Hillis1

Bridging AI and Cognitive Science workshop, ICLR April 24th, 2020

1 2

slide-2
SLIDE 2
  • 2

An everyday problem…

…where are the keys?

slide-3
SLIDE 3

Resource allocation in visual search

  • 3
  • Main contribution: frame visual search as a reinforcement learning

problem

  • Fixations as information-gathering actions
  • Do people employ optimal strategies?
slide-4
SLIDE 4

Resource allocation in visual search

  • 4
  • Main contribution: frame visual search as a reinforcement learning

problem

  • Fixations as information-gathering actions
  • Do people employ optimal strategies?
  • Challenges:
  • Representing the state space — world is high-dimensional; what

features does visual system have access to?

  • Finding the optimal policy — reward function is sparse; how to

balance cost of sampling and performance?

slide-5
SLIDE 5

Naturalistic visual search in VR

  • VR + gaze tracking, fixed camera location
  • Cluttered room, 1 target among many distractors
  • “Find the target within 8 seconds”
  • 6 different rooms x 5 locations per room x 10 trials per location = 300 unique scenes
  • Some trials assisted
  • 5
slide-6
SLIDE 6
  • 6
slide-7
SLIDE 7
  • 7
slide-8
SLIDE 8

Start End

  • 8
slide-9
SLIDE 9

Callaway & Griffiths 2018

Meta-level Markov Decision Process

  • 9
slide-10
SLIDE 10

Callaway & Griffiths 2018

Meta-level Markov Decision Process

  • 10
  • Latent: {Ftrue, itrue}
  • Scene features and target identity unknown to the agent
  • States: {F

, J, ftarget}

  • Mean and precision of each feature for each object
  • Actions: {o, ⊥}
  • Fixate on object o, or terminate
  • Transitions: measure X ~ N(Ftrue, Jmeas)
  • Jmeas decreases with distance from o
  • Integrate X into F and J with Bayesian cue combination
  • Rewards: if fixating o then R = -c; if ⊥ then R = 1 if argmax(P(target | F

, J)) = itrue and 0 otherwise

  • Reward agent when most probable target given state matches true target
slide-11
SLIDE 11

Callaway & Griffiths 2018

Meta-level Markov Decision Process

  • 11
  • Latent: {Ftrue, itrue}
  • Scene features and target identity unknown to the agent
  • States: {F

, J, ftarget}

  • Mean and precision of each feature for each object
  • Actions: {o, ⊥}
  • Fixate on object o, or terminate
  • Transitions: measure X ~ N(Ftrue, Jmeas)
  • Jmeas decreases with distance from o
  • Integrate X into F and J with Bayesian cue combination
  • Rewards: if fixating o then R = -c; if ⊥ then R = 1 if argmax(P(target | F

, J)) = itrue and 0 otherwise

  • Reward agent when most probable target given state matches true target
slide-12
SLIDE 12

Callaway & Griffiths 2018

Meta-level Markov Decision Process

  • 12
  • Latent: {Ftrue, itrue}
  • Scene features and target identity unknown to the agent
  • States: {F

, J, ftarget}

  • Mean and precision of each feature for each object
  • Actions: {o, ⊥}
  • Fixate on object o, or terminate
  • Transitions: measure X ~ N(Ftrue, Jmeas)
  • Jmeas decreases with distance from o
  • Integrate X into F and J with Bayesian cue combination
  • Rewards: if fixating o then R = -c; if ⊥ then R = 1 if argmax(P(target | F

, J)) = itrue and 0 otherwise

  • Reward agent when most probable target given state matches true target
slide-13
SLIDE 13

Callaway & Griffiths 2018

Meta-level Markov Decision Process

  • 13
  • Latent: {Ftrue, itrue}
  • Scene features and target identity unknown to the agent
  • States: {F

, J, ftarget}

  • Mean and precision of each feature for each object
  • Actions: {o, ⊥}
  • Fixate on object o, or terminate
  • Transitions: measure X ~ N(Ftrue, Jmeas)
  • Jmeas decreases with distance from o
  • Integrate X into F and J with Bayesian cue combination
  • Rewards: if fixating o then R = -c; if ⊥ then R = 1 if argmax(P(target | F

, J)) = itrue and 0 otherwise

  • Reward agent when most probable target given state matches true target
slide-14
SLIDE 14

Callaway & Griffiths 2018

Meta-level Markov Decision Process

  • 14
  • Latent: {Ftrue, itrue}
  • Scene features and target identity unknown to the agent
  • States: {F

, J, ftarget}

  • Mean and precision of each feature for each object
  • Actions: {o, ⊥}
  • Fixate on object o, or terminate
  • Transitions: measure X ~ N(Ftrue, Jmeas)
  • Jmeas decreases with distance from o
  • Integrate X into F and J with Bayesian cue combination
  • Rewards: if fixating o then R = -c; if ⊥ then R = 1 if argmax(P(target | F

, J)) = itrue and 0 otherwise

  • Reward agent when most probable target given state matches true target
slide-15
SLIDE 15

Challenge I: representing the belief space Challenge II: finding the optimal policy

  • 15
slide-16
SLIDE 16

Which features to include?

  • 16
slide-17
SLIDE 17

Which features to include?

Treisman & Gelade, 1980 Horowitz & Wolfe, 2017

  • 17

Target Object attributes

Shape Color

Objects

slide-18
SLIDE 18
  • 18

Which features to include?

slide-19
SLIDE 19

D2 distribution 3D mesh

Shape

  • 19

Which features to include?

slide-20
SLIDE 20

D2 distribution 3D mesh

Shape

CIELAB A B A B 2D texture

Color

  • 20

Which features to include?

slide-21
SLIDE 21

D2 distribution 3D mesh

Shape

CIELAB A B A B 2D texture

Color

PCA PCA

  • 21

Which features to include?

slide-22
SLIDE 22

D2 distribution 3D mesh

Shape

CIELAB A B A B 2D texture

Color

PCA PCA

Full Partial (3PCs) Full Partial (3PCs) Similarity structure

  • 22

Which features to include?

slide-23
SLIDE 23

Shape and color predict gaze

  • 23
slide-24
SLIDE 24

Shape and color predict gaze

Gaze on objects Gaze on objects

  • 24
slide-25
SLIDE 25

Shape and color predict gaze

  • 25
slide-26
SLIDE 26

Challenge I: representing the belief space Challenge II: finding the optimal policy

  • 26
slide-27
SLIDE 27

Calculate/update posterior probabilities If maximum exceeds criterion, STOP Move eyes to object most likely to be target Sample information at fixated location

Najemnik and Geisler, 2005 Yang, Lengyel and Wolpert, 2017

“Ideal observer” model of visual search

  • 27
  • Can be expressed as a policy in the meta-MDP

, but not necessarily optimal

slide-28
SLIDE 28

Dense Layers π V Policy Value

Object locations Object features Target features Posterior

Input

  • Proximal Policy Optimization

(PPO, Schulman, 2017), implemented with tf-agents

  • 10 replications, manually tuned

hyper-parameters

  • Manual tweaking of input

representation & initialization

  • 28

Optimizing meta-level return with deep reinforcement learning

slide-29
SLIDE 29

Optimizing meta-level return with deep reinforcement learning

  • Proximal Policy Optimization

(PPO, Schulman, 2017), implemented with tf-agents

  • 10 replications, manually tuned

hyper-parameters

  • Manual tweaking of input

representation & initialization

  • 29

0.5 1 1.5 2

6imulated eSisRdes (milliRns)

0.2 0.4 0.6 0.8

5eward

slide-30
SLIDE 30

Optimizing meta-level return with deep reinforcement learning

  • Proximal Policy Optimization

(PPO, Schulman, 2017), implemented with tf-agents

  • 10 replications, manually tuned

hyper-parameters

  • Manual tweaking of input

representation & initialization

  • 30

0.5 1 1.5 2

6imulated eSisRdes (milliRns)

0.2 0.4 0.6 0.8

5eward

NG

slide-31
SLIDE 31

Model

Start End

Human

Does optimal policy match humans?

  • 31
slide-32
SLIDE 32

Model

Start End

Human

Object Object

Does optimal policy match humans?

  • 32
slide-33
SLIDE 33

Model

Start End

Human

Which features drive human search?

  • 33
slide-34
SLIDE 34

Ongoing work

  • 34
  • Alternative schemes for extracting low-dimensional feature

representations of objects

  • Deep convolutional neural network models of human ventral

visual stream (Yamins et al. 2014, Fan et al. 2019)

  • MeshNet model of 3D shape representation (Feng et al. 2018)
slide-35
SLIDE 35

Ongoing work

  • 35
  • Alternative schemes for extracting low-dimensional feature

representations of objects

  • Deep convolutional neural network models of human ventral

visual stream (Yamins et al. 2014, Fan et al. 2019)

  • MeshNet model of 3D shape representation (Feng et al. 2018)
  • Investigating the learned policy
  • Is it optimal?
slide-36
SLIDE 36
  • 36

Thank you!