Descriptors III CSE 576 Ali Farhadi Many slides from Larry - - PowerPoint PPT Presentation

▶
descriptors iii
SMART_READER_LITE
LIVE PREVIEW

Descriptors III CSE 576 Ali Farhadi Many slides from Larry - - PowerPoint PPT Presentation

Mar 22, 2024 50 likes •531 views

Descriptors III CSE 576 Ali Farhadi Many slides from Larry Zitnick, Steve Seitz How can we find corresponding points? How can we find correspondences? SIFT descriptor Full version Divide the 16x16 window into a 4x4 grid of cells (2x2 case

slide-1
SLIDE 1

Descriptors III

CSE 576

Ali Farhadi Many slides from Larry Zitnick, Steve Seitz

slide-2
SLIDE 2

How can we find corresponding points?

slide-3
SLIDE 3

How can we find correspondences?

slide-4
SLIDE 4

SIFT descriptor

Full version

  • Divide the 16x16 window into a 4x4 grid of cells (2x2 case shown below)
  • Compute an orientation histogram for each cell
  • 16 cells * 8 orientations = 128 dimensional descriptor

Adapted from slide by David Lowe

slide-5
SLIDE 5

Local Descriptors: Shape Context

Count the number of points inside each bin, e.g.: Count = 4 Count = 10 ... Log-polar binning: more precision for nearby points, more flexibility for farther points.

Belongie & Malik, ICCV 2001

  • K. Grauman, B. Leibe
slide-6
SLIDE 6

Bag of Words

…..

frequency

codewords

slide-7
SLIDE 7

Another Representation: Filter bank

slide-8
SLIDE 8

level 0 level 1 level 2 Lazebnik, Schmid & Ponce (CVPR 2006)

Spatial pyramid representation

  • Extension of a bag of features
  • Locally orderless representation at several levels of resolution
slide-9
SLIDE 9

What about Scenes?

slide-10
SLIDE 10

Demo : Rapid image understanding

Instructions: 9 photographs will be shown for half a second each. Your task is to memorize these pictures 
 


By Aude Oliva

slide-11
SLIDE 11

Credit: A. Torralba

slide-12
SLIDE 12

Credit: A. Torralba

slide-13
SLIDE 13

Credit: A. Torralba

slide-14
SLIDE 14

Credit: A. Torralba

slide-15
SLIDE 15

Credit: A. Torralba

slide-16
SLIDE 16

Credit: A. Torralba

slide-17
SLIDE 17

Credit: A. Torralba

slide-18
SLIDE 18

Credit: A. Torralba

slide-19
SLIDE 19

Credit: A. Torralba

slide-20
SLIDE 20
slide-21
SLIDE 21


 Which of the following pictures have you seen ?
 
 If you have seen the image
 clap your hands once
 


Memory Test

Credit: A. Torralba

slide-22
SLIDE 22

Have you seen this picture ?

Credit: A. Torralba

slide-23
SLIDE 23

NO

Credit: A. Torralba

slide-24
SLIDE 24

Have you seen this picture ?

Credit: A. Torralba

slide-25
SLIDE 25

NO

slide-26
SLIDE 26

Have you seen this picture ?

slide-27
SLIDE 27

NO

slide-28
SLIDE 28

Have you seen this picture ?

slide-29
SLIDE 29

NO

Credit: A. Torralba

slide-30
SLIDE 30

Have you seen this picture ?

slide-31
SLIDE 31

Yes

Credit: A. Torralba

slide-32
SLIDE 32

Have you seen this picture ?

Credit: A. Torralba

slide-33
SLIDE 33

NO

Credit: A. Torralba

slide-34
SLIDE 34

You have seen these pictures You were tested with these pictures

slide-35
SLIDE 35

The gist of the scene

In a glance, we remember the meaning of an image and its global layout but some objects and details are forgotten

slide-36
SLIDE 36

Holistic scene representation: Shape of a scene

  • Finding a low-dimensional

“scene space”

  • Clustering by humans
  • Split images into groups
  • ignore objects, categories
slide-37
SLIDE 37

Spatial envelope properties

  • Naturalness
  • natural vs. man-made environments
slide-38
SLIDE 38

Spatial envelope properties

  • Openness
  • decreases as number of boundary elements

increases

slide-39
SLIDE 39

Spatial envelope properties

  • Roughness
  • size of elements at each spatial scale, related to

fractal dimension

slide-40
SLIDE 40

Spatial envelope properties

  • Expansion (man-made environments)
  • depth gradient of the space
slide-41
SLIDE 41

Spatial envelope properties

  • Ruggedness (natural environments)
  • deviation of ground relative to horizon
slide-42
SLIDE 42

Scene statistics

  • DFT (energy spectrum)
  • throw out phase function (represents local properties)
  • Windowed DFT (spectrogram)
  • Coarse local information
  • 8x8 grid for these results
slide-43
SLIDE 43

Scene statistics

slide-44
SLIDE 44

Scene classification from statistics

  • Different scene categories have different spectral

signatures

  • Amplitude captures roughness
  • Orientation captures dominant edges
slide-45
SLIDE 45

Learning the spatial envelope

  • Use linear regression to learn
  • DST (discriminant spectral template)
  • WDST (windowed discriminant spectral template)
  • Relate spectral representation to each spatial

envelope feature

slide-46
SLIDE 46

Gist descriptor

8 orientations 4 scales x 16 bins 512 dimensions

Similar to SIFT (Lowe 1999) applied to the entire image

  • M. Gorkani, R. Picard, ICPR 1994; Walker, Malik. Vision Research 2004; Vogel et al. 2004;

Fei-Fei and Perona, CVPR 2005; S. Lazebnik, et al, CVPR 2006; …

Oliva and Torralba, 2001

slide-47
SLIDE 47

Example visual gists

Oliva & Torralba (2001)

slide-48
SLIDE 48

Features

▪ Where: ▪ Interest points ▪ Corners ▪ Blobs ▪ Grid ▪ Spatial Pyramids ▪ Global ▪ What: (Descriptors) ▪ Sift, HOG ▪ Shape Context ▪ Bag of words ▪ Filter banks