[PPT] - Efficient Superpixel Creation in High-resolution Images by Applying PowerPoint Presentation

SLIDE 1

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT

Master’s Thesis Jan Draegert

Freie Universit¨ at Berlin Fachbereich Mathematik und Informatik Institut f¨ ur Informatik

August 4, 2017

SLIDE 2

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT

SLIDE 3

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT

SLIDE 4

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT

SLIDE 5

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT

SLIDE 6

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT

SLIDE 7

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT

Outline

Superpixels PLANT Experiments Application to Vision of Soccer Robots

SLIDE 8

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Superpixels

Image Segmentation

SLIDE 9

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Superpixels

Superpixels

SLIDE 10

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

Partition-Locating, Axis-aligned, and Node-queing Tree

Generating a PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

SLIDE 11

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split location ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 12

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split location ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 13

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 14

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Split Location

◮ pivot ← xmin + width/2 ◮ step width ← width/4 ◮ while step width ≥ 1 do:

◮ if

>
◮ xmax ← pivot

◮ pivot ← pivot − step width ◮ else ◮ xmin ← pivot ◮ pivot ← pivot + step width ◮ step width ← step width/2

◮ return pivot

SLIDE 15

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Integral Image

I Precalculation II Accessing an area

SLIDE 16

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Integral Image

I Precalculation

◮ simple:

IΣ(x, y) =

i≤x

i=0

j≤y

j=0

I(i, j)

SLIDE 17

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Integral Image

I Precalculation

◮ simple: IΣ(x, y) = i≤x i=0

j≤y

j=0 I(i, j) ◮ fast:

S(x, y) = S(x − 1, y) + I(x, y) IΣ(x, y) = IΣ(x, y − 1) + S(x, y)

SLIDE 18

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Integral Image

I Precalculation II Accessing an area

x0≤x≤x1

y0≤y≤y1

I(x, y) = IΣ(D) + IΣ(A) − IΣ(B) − IΣ(C) where: A = (x0, y0) B = (x1, y0) C = (x0, y1) D = (x1, y1)

SLIDE 19

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Split Location

◮ pivot ← xmin + width/2 ◮ step width ← width/4 ◮ while step width ≥ 1 do:

◮ if

>
◮ xmax ← pivot

◮ pivot ← pivot − step width ◮ else ◮ xmin ← pivot ◮ pivot ← pivot + step width ◮ step width ← step width/2

◮ return pivot

SLIDE 20

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 21

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 22

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Split Direction

◮ if (∆h ≥ ∆v)

◮ split horizontally

◮ else

◮ split vertically

∆h and ∆v are computed while split location is searched

SLIDE 23

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Split Direction

◮ pivot ← xmin + width/2, step width ← width/4 ◮ ∆h ← 0 ◮ while step width ≥ 1 do:

◮ if

>
◮ xmax ← pivot

◮ pivot ← pivot − step width ◮ ∆h ← ∆h + step width ·

◮ else

◮ xmin ← pivot ◮ pivot ← pivot + step width ◮ ∆h ← ∆h + step width ·

◮ step width ← step width/2

◮

SLIDE 24

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 25

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 26

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Priority

◮ if (∆h ≥ ∆v)

◮ split horizontally ◮ set priority to ∆h

◮ else

◮ split vertically ◮ set priority to ∆v

SLIDE 27

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 28

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 29

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

Termination Criterion

◮ Stop if

◮ max number of leaves reached ◮ or priority < θ

SLIDE 30

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 31

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT Algorithm

PLANT

◮ Initially: PLANT and priority queue are empty ◮ Root node contains whole image ◮ Insert() root node ◮ While not terminated:

◮ Grow()

Insert()

◮ Find split locations ◮ Determine split direction ◮ Calculate priority ◮ Add to priority queue

and PLANT

Grow()

◮ Pop node with highest

priority from queue

◮ Split node ◮ For both resulting nodes:

◮ Insert()

SLIDE 32

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT PLANT PLANT Superpixels

PLANT Superpixels

Efficient PLANT-based Superpixel Algorithms

◮ PLANT: leaf ˆ

= superpixel

◮ O(n + k · log(n))

◮ PLANTm: merge leaves to superpixel

◮ O(n + k · log(n) + k2)

◮ #superpixels (k) ≪ #pixels (n) ◮ Fast computation of asymptotic most expensive term

SLIDE 33

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Run times

Experiments

105 106 101 102 103 104 image resolution (pixel) time (ms) QS ERS LSC ERGC reSEEDS SLIC SEEDS FH PB MSS CW ETPS preSLIC PF W PLANTm PLANT

SLIDE 34

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Run times

Experiments

105 106 101 102 103 104 image resolution (pixel) time (ms) ERS ERSf LSC LSCf MSS MSSf reSEEDS reSEEDSf ETPS preSLIC preSLICf SEEDS SEEDSf SLIC SLICf

SLIDE 35

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Run times

Experiments

105 106 107 100 101 102 103 30 Hz HD resolution image resolution (pixel) time (ms) PF ETPS SEEDSf preSLIC preSLICf SLICf W PLANTm PLANT

SLIDE 36

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Run times

Experiments

2 4 6 8 ·106 500 1,000 1,500 image resolution (pixel) time (ms) PF ETPS SEEDSf preSLIC preSLICf SLICf W PLANTm PLANT

SLIDE 37

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Run times

Experiments

105 106 107 500 1,000 1,500 image resolution (pixel) time (ms) PF ETPS SEEDSf preSLIC preSLICf SLICf W PLANTm PLANT

SLIDE 38

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Run times

Experiments

105 106 107 10 20 30 image resolution (pixel) time (ms) PLANTm merge PLANTm generate PLANTm convert PLANTm total PLANT generate PLANT convert PLANT total

SLIDE 39

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Run times

Experiments

2 4 6 8 ·106 10 20 30 image resolution (pixel) time (ms) PLANTm merge PLANTm generate PLANTm convert PLANTm total PLANT generate PLANT convert PLANT total

SLIDE 40

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Quality

0.2 0.4 0.6 0.8 1 ·104 0.4 0.6 0.8 1 number of superpixels boundary recall 0.2 0.4 0.6 0.8 1 ·104 0.2 0.4 0.6 number of superpixels undersegmentation error 0.2 0.4 0.6 0.8 1 ·104 0.4 0.6 0.8 1 number of superpixels explained variation PLANT ERS PLANTm FH W PB CW ERGC SLIC PF preSLIC QS SEEDS SEEDSf reSEEDS SLICf ETPS preSLICf MSS LSC

SLIDE 41

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Quality

Experiments

0.2 0.4 0.6 0.8 1 ·104 0.4 0.6 0.8 1 number of superpixels explained variation PLANT ERS PLANTm FH W PB CW ERGC SLIC PF preSLIC QS SEEDS SEEDSf reSEEDS SLICf ETPS preSLICf MSS LSC

SLIDE 42

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Experiments Quality

INPUT ETPS PLANT SLIC PLANTd W PLANTm SEEDS PLANTmd FH

SLIDE 43

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Field Contour Extraction

Field Contour Extraction

SLIDE 44

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Field Contour Extraction

Field Contour Extraction

SLIDE 45

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Field Contour Extraction

Field Contour Extraction

SLIDE 46

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

White Blob Extraction

SLIDE 47

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

Ball Candidates Detection

SLIDE 48

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

Ball Candidates Detection

SLIDE 49

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

Ball Candidates Detection

SLIDE 50

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

Goalposts Detection

SLIDE 51

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

Goalposts Detection

SLIDE 52

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

Goalposts Detection

SLIDE 53

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Application to Vision of Soccer Robots Object Detection

SLIDE 54

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Conclusion

Conclusion

◮ PLANT introduced ◮ PLANT and PLANTm superpixels

algorithms proposed

◮ Experiments showed efficiency ◮ Vision for soccer robots

implemented

SLIDE 55

Efficient Superpixel Creation in High-resolution Images by Applying a PLANT Conclusion