Edge Detection State of The Art P. Dollar and C. Zitnick - - PowerPoint PPT Presentation

▶
edge detection state of the art
SMART_READER_LITE
LIVE PREVIEW

Edge Detection State of The Art P. Dollar and C. Zitnick - - PowerPoint PPT Presentation

May 12, 2023 387 likes •746 views

Edge Detection State of The Art P. Dollar and C. Zitnick Structured Forests for Fast Edge Detection ICCV 2013 Code: http://research.microsoft.com/en-us/downloads/ 389109f6-b4e8-404c-84bf-239f7cbf4e3d/default.aspx (Time stamp: Sept 15, 2014)

slide-1
SLIDE 1

Edge Detection State of The Art

  • P. Dollar and C. Zitnick

Structured Forests for Fast Edge Detection ICCV 2013

Code: http://research.microsoft.com/en-us/downloads/ 389109f6-b4e8-404c-84bf-239f7cbf4e3d/default.aspx

(Time stamp: Sept 15, 2014)

Sanja Fidler CSC420: Intro to Image Understanding 1 / 63

slide-2
SLIDE 2

Testing the Canny Edge Detector

Let’s take this image Our goal (a few lectures from now) is to detect objects (cows here)

Sanja Fidler CSC420: Intro to Image Understanding 2 / 63

slide-3
SLIDE 3

Testing the Canny Edge Detector

image gradients + NMS Canny’s edges

Sanja Fidler CSC420: Intro to Image Understanding 3 / 63

slide-4
SLIDE 4

Testing the Canny Edge Detector

image gradients + NMS Canny’s edges

Sanja Fidler CSC420: Intro to Image Understanding 3 / 63

slide-5
SLIDE 5

Testing the Canny Edge Detector

image gradients + NMS Canny’s edges Lots of “distractor” and missing edges Can we do better?

Sanja Fidler CSC420: Intro to Image Understanding 3 / 63

slide-6
SLIDE 6

Annotate...

Imagine someone goes and annotates which edges are correct ... and someone has:

Sanja Fidler CSC420: Intro to Image Understanding 4 / 63

slide-7
SLIDE 7

Annotate...

Imagine someone goes and annotates which edges are correct ... and someone has:

The Berkeley Segmentation Dataset and Benchmark

by D. Martin and C. Fowlkes and D. Tal and J. Malik

Sanja Fidler CSC420: Intro to Image Understanding 4 / 63

slide-8
SLIDE 8

... and do Machine Learning

How can we make use of such data to improve our edge detector?

Sanja Fidler CSC420: Intro to Image Understanding 5 / 63

slide-9
SLIDE 9

... and do Machine Learning

How can we make use of such data to improve our edge detector? We can use Machine Learning techniques to:

Train classifiers!

Please learn what a classifier /classification is In particular, learn what a Support Vector Machine (SVM) is (some links to tutorials are on the class webpage) With each week it’s going to be more important to know about this You don’t need to learn all the details / math, but to understand the concept enough to know what’s going on

Sanja Fidler CSC420: Intro to Image Understanding 6 / 63

slide-10
SLIDE 10

... and do Machine Learning

How can we make use of such data to improve our edge detector? We can use Machine Learning techniques to:

Train classifiers!

Please learn what a classifier /classification is In particular, learn what a Support Vector Machine (SVM) is (some links to tutorials are on the class webpage) With each week it’s going to be more important to know about this You don’t need to learn all the details / math, but to understand the concept enough to know what’s going on

Sanja Fidler CSC420: Intro to Image Understanding 6 / 63

slide-11
SLIDE 11

Classification – a Disney edition (pictures only)

Each data point x lives in a n-dimensional space, x ∈ Rn We have a bunch of data points xi, and for each we have a label, yi A label yi can be either 1 (positive example – correct edge in our case), or −1 (negative example – wrong edge in our case)

Sanja Fidler CSC420: Intro to Image Understanding 7 / 63

slide-12
SLIDE 12

Classification – a Disney edition (pictures only)

Sanja Fidler CSC420: Intro to Image Understanding 8 / 63

slide-13
SLIDE 13

Classification – a Disney edition (pictures only)

Sanja Fidler CSC420: Intro to Image Understanding 8 / 63

slide-14
SLIDE 14

Classification – a Disney edition (pictures only)

Sanja Fidler CSC420: Intro to Image Understanding 8 / 63

slide-15
SLIDE 15

Training an Edge Detector

How should we do this?

Sanja Fidler CSC420: Intro to Image Understanding 9 / 63

slide-16
SLIDE 16

Training an Edge Detector

How should we do this?

Sanja Fidler CSC420: Intro to Image Understanding 9 / 63

slide-17
SLIDE 17

Training an Edge Detector

We extract lots of image patches

Sanja Fidler CSC420: Intro to Image Understanding 9 / 63

slide-18
SLIDE 18

Training an Edge Detector

We extract lots of image patches These are our training data

Sanja Fidler CSC420: Intro to Image Understanding 9 / 63

slide-19
SLIDE 19

Training an Edge Detector

We extract lots of image patches These are our training data We convert each image patch P (a matrix) into a vector x

Sanja Fidler CSC420: Intro to Image Understanding 9 / 63

slide-20
SLIDE 20

Training an Edge Detector

We extract lots of image patches These are our training data We convert each image patch P (a matrix) into a vector x Well... This works better: Extract image features for each patch

Sanja Fidler CSC420: Intro to Image Understanding 9 / 63

slide-21
SLIDE 21

Training an Edge Detector

We extract lots of image patches These are our training data We convert each image patch P (a matrix) into a vector x Well... This works better: Extract image features for each patch Image features are mappings from images (or patches) to other (vector) meaningful representations. More on this in the next class!

Sanja Fidler CSC420: Intro to Image Understanding 9 / 63

slide-22
SLIDE 22

Using an Edge Detector

Once trained, how can we use our new edge detector?

Sanja Fidler CSC420: Intro to Image Understanding 10 / 63

slide-23
SLIDE 23

Using an Edge Detector

We extract all image patches

Sanja Fidler CSC420: Intro to Image Understanding 10 / 63

slide-24
SLIDE 24

Using an Edge Detector

We extract all image patches Extract features and use our trained classifier

Sanja Fidler CSC420: Intro to Image Understanding 10 / 63

slide-25
SLIDE 25

Using an Edge Detector

We extract all image patches Extract features and use our trained classifier Place the predicted value (score) in the output matrix

Sanja Fidler CSC420: Intro to Image Understanding 10 / 63

slide-26
SLIDE 26

Comparisons: Canny vs Structured Edge Detector

image image gradients gradients + NMS “edgeness score” score + NMS

Sanja Fidler CSC420: Intro to Image Understanding 11 / 63

slide-27
SLIDE 27

Comparisons: Canny vs Structured Edge Detector

image image gradients gradients + NMS “edgeness” score score + NMS image gradient “edgeness” score

Sanja Fidler CSC420: Intro to Image Understanding 12 / 63

slide-28
SLIDE 28

Comparisons: Canny vs Structured Edge Detector

image image gradients gradients + NMS “edgeness” score score + NMS

Sanja Fidler CSC420: Intro to Image Understanding 13 / 63

slide-29
SLIDE 29

Comparisons: Canny vs Structured Edge Detector

image image gradients gradients + NMS “edgeness” score score + NMS

Sanja Fidler CSC420: Intro to Image Understanding 14 / 63

slide-30
SLIDE 30

Comparisons: Canny vs Structured Edge Detector

image image gradients gradients + NMS “edgeness” score score + NMS image gradient “edgeness” score

Sanja Fidler CSC420: Intro to Image Understanding 15 / 63

slide-31
SLIDE 31

Evaluation

Figure: green=correct, blue=wrong, red=missing, green+blue=output edges

Sanja Fidler CSC420: Intro to Image Understanding 16 / 63

slide-32
SLIDE 32

Evaluation

Recall: How many of all annotated edges we got correct (best is 1) Precision How many of all output edges we got correct (best is 1) Recall = # of green (correct edges) # of all edges in ground-truth (second picture)

Sanja Fidler CSC420: Intro to Image Understanding 16 / 63

slide-33
SLIDE 33

Evaluation

Recall: How many of all annotated edges we got correct (best is 1) Precision How many of all output edges we got correct (best is 1) Precision = # of green (correct edges) # of all edges in output (second picture)

Sanja Fidler CSC420: Intro to Image Understanding 16 / 63

slide-34
SLIDE 34

Evaluation

Recall: How many of all annotated edges we got correct (best is 1) Precision How many of all output edges we got correct (best is 1)

Sanja Fidler CSC420: Intro to Image Understanding 16 / 63

slide-35
SLIDE 35

Lesson 1

Trained detectors (typically) perform better (true for all applications) In this case, the method seems to work better for finding object boundaries (edges) than finding text boundaries. Any idea why? What would you do if you wanted to detect text (e.g., licence plates)? Think about your problem, don’t just use code as a black box

Sanja Fidler CSC420: Intro to Image Understanding 17 / 63