Processing Megapixel Images with Deep Attention-Sampling Models - - PowerPoint PPT Presentation

▶

Oct 04, 2022 242 likes •394 views

Processing Megapixel Images with Deep Attention-Sampling Models Angelos Katharopoulos & Fran cois Fleuret ICML, June 11, 2019 Funded by How do DNNs process large images? Cropping and downsampling to a manageable resolution (e.g. 224

SLIDE 1

Processing Megapixel Images with Deep Attention-Sampling Models

Angelos Katharopoulos & Fran¸ cois Fleuret ICML, June 11, 2019

Funded by

SLIDE 2

How do DNNs process large images?

Cropping and downsampling to a manageable resolution (e.g. 224 × 224) Dividing the image into patches and processing them separately

∗image taken from the Imagenet dataset

A. Katharopoulos

Deep Attention-Sampling Models 2/9

SLIDE 3

Our contributions

◮ Disentangle the computational and memory requirements from the input

resolution.

◮ Sample from a soft attention to only process a fraction of the image in high

resolution.

◮ We derive gradients through the sampling for all parameters and train our models

end-to-end.

A. Katharopoulos

Deep Attention-Sampling Models 3/9

SLIDE 4

Soft Attention

Given an input x we define a neural network Ψ(x) that uses attention Ψ(x) = g K

a(x)if (x)i

= g
EI∼a(x)[f (x)I]
,

where f (x) ∈ RK×D are the features and a(x) ∈ RK

+ is the attention distribution.

A. Katharopoulos

Deep Attention-Sampling Models 4/9

SLIDE 5

Attention Sampling

We approximate Ψ(x) by Monte Carlo Ψ(x) ≈ g   1 N

q∈Q

f (x)q   where Q = {qi ∼ a(x) | i ∈ {1, 2, . . . , N}}. We show that

◮ Sampling from the attention is optimal to approximate Ψ(x) if

f (x)i = f (x)j ∀ i, j

◮ We can compute the gradients both for the parameters a(·) and f (·)

A. Katharopoulos

Deep Attention-Sampling Models 5/9

SLIDE 6