School of EECS Washington State University Artificial Intelligence - - PowerPoint PPT Presentation

School of EECS Washington State University

1 Artificial Intelligence

Artificial Intelligence 2

• I see a cougar.
• I see two cougars?
• I see water and grass.
• Pretty tree.
• Distance to cougar: 10m.
• Should I leave?
• Animal, cat, cougar;

moving left.

• Person, police, gun;

moving right.

• Trash bin.
• Danger!
• Can’t park here.

Scene Understanding

} Edge detection } Shape detection } Motion: optical flow, tracking } Object detection } Image classification } Scene understanding

Artificial Intelligence 3

Artificial Intelligence 4

} Given image intensity I(x,y) (e.g., grayscale) } Step 1: Smooth image

• Gaussian Nσ(x,y)
• Replace intensity I(x,y) with I * Nσ (i.e., convolve)

Artificial Intelligence 5

} Step 2: Find intensity gradient G

• Convolve image with Sobel operator
• 𝐻" =

−1 +1 −2 +2 −1 +1 ∗ I 𝐻+ = −1 −2 −1 +1 +2 +1 ∗ I

• 𝐻 =

𝐻"

, +𝐻+ ,

𝜄 = tan12

34 35

Artificial Intelligence 6

255 255 255 255 255 255

Gx = 255+510+0 = 765 Gy = -255+0+1020 = 765 G = 1082 θ = tan-1(765/765) = 45°

} Step 3: Thin edges

Artificial Intelligence 7

A B C edge gradient If B < max(A,B,C) Then B = 0

255 255 255 4 5 3 255 255 255 255 255 255 255 255 255

} Step 4: Apply double threshold with

hysteresis

Artificial Intelligence 8

Image G min value max value removed removed kept kept kept: if next to kept then kept kept

Artificial Intelligence 9

Artificial Intelligence 10

} Detect simple parametric shapes

• Lines, circles, etc.

} Noise tolerant } Approach

• For each edge

– Increment models consistent with edge

• Choose models with most votes

Artificial Intelligence 11

Artificial Intelligence 12

Model: y = mx + b

1 1 1 1 3 1 1 1 1

Image Parameter Space m b 1 x y

} Parameter space (m, b) infinite

• How to choose range and increment?

} Another model of a line:

Artificial Intelligence 13

r = x cosθ + y sinθ

Artificial Intelligence 14

Probabilistic Hough Transform: Line segments

Artificial Intelligence 15

Threshold > 90 Threshold > 100

Artificial Intelligence 16

} Assumptions

• Brightness constancy
• Small motion

Artificial Intelligence 17

} Frame differencing

Artificial Intelligence 18

Artificial Intelligence 19

Artificial Intelligence 20

Have Ix, Iy, It. Solve for u and v. One more assumption: Constant flow. E.g., the 3x3 patch around a pixel have same

• displacement. [Lucas-Kinkade Method]

Artificial Intelligence 21

} Given an image feature to track

• E.g., bounding box

} Find it as the image changes

Artificial Intelligence 22

} As an image alignment problem...

Artificial Intelligence 23

I(x) T(x) W(x;p)

p involves translation, rotation, scaling

Artificial Intelligence 24

What if we don't have a template...?

} Approach #1: Feature-based

• Define various image features
• Model object in terms of these features
• Look for feature-level matches in image

} Approach #2: Network-based

• Train a deep neural network on lots of images with known
• bjects in known locations
• Use network to locate objects in an image

Artificial Intelligence 25

} Features

• Histograms of Oriented Gradients (HOG)

Artificial Intelligence 26

} Pedestrian

Artificial Intelligence 27

HOG

Artificial Intelligence 28

} Features

• Scale Invariant Feature Transform (SIFT)

Artificial Intelligence 29

• 1. Find extreme

points

• 2. Discard low-

contrast points

• 3. Filter points
• n edges

"keypoints"

Artificial Intelligence 30

http://weitz.de/sift/

Artificial Intelligence 31

HOG/SIFT features Machine Learning

airplane bicylcle truck

. . . Training: Testing:

Artificial Intelligence 32

Need a lot of training data... Figures out features automatically

} PASCAL Visual Object Classes (VOC)

• host.robots.ox.ac.uk/pascal/VOC
• 20,000 images
• 20 classes

} ImageNet

• image-net.org
• 1.5M images
• 200 classes

Artificial Intelligence 33

Artificial Intelligence 34

ReLU

} CIFAR-10

• 60,000 images
• 10 classes

} CIFAR-100

• 60,000 images
• 100 classes

Artificial Intelligence 35

https://www.cs.toronto.edu/~kriz/cifar.html

Artificial Intelligence 36

I see an astronaut walking on

• grass. There is a bright gold

dog bowl nearby. There are several puddles of water, some with wavy lines above them. There are brown bumps on the grass that are smelly. There is a happy alien with one eye. location(agent,1,2)

• rientation(agent,right)

location(wumpus,4,2) location(gold,2,3) location(pit,3,1): 1.0 location(pit,3,3): 0.8 dimensions(4,4) bestAction(goforward)

?

Artificial Intelligence 37

MIT Places Uses 16-layer ConvNet

http://places2.csail.mit.edu/demo.html

} Drag image here } Edit Alt Text...

Artificial Intelligence 38

"A person walking a dog on a leash in front of a building."

Artificial Intelligence 39

[Xiao et al., ECCV 2018]

} OpenCV (opencv.org) } scikit-image (scikit-image.org)

Artificial Intelligence 40

} Techniques depend on goals of vision

• Edge detection
• Shape detection
• Motion: optical flow,

tracking

• Object detection
• Image classification

} Future

• Scene understanding
• Video summarization
• Fake images and video

Artificial Intelligence 41