Open tools and methods for large scale segmentation of Very High - - PowerPoint PPT Presentation

▶
open tools and methods for large scale segmentation of
SMART_READER_LITE
LIVE PREVIEW

Open tools and methods for large scale segmentation of Very High - - PowerPoint PPT Presentation

Dec 08, 2022 285 likes •593 views

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion Open tools and methods for large scale segmentation of Very High Resolution satellite images Julien Michel

slide-1
SLIDE 1

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Open tools and methods for large scale segmentation of Very High Resolution satellite images

Julien Michel (CNES), Manuel Grizonnet (CNES), Arnaud Jaen (CS-SI), Sebastien Harasse (CS-SI), Luc Hermitte (CS-SI), Jonathan Guinet (CS-SI), Julien Malik (CS-SI), Mickaël Savinaud (CS-SI) November 19, 2012

slide-2
SLIDE 2

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Outline

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

slide-3
SLIDE 3

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Outline

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

slide-4
SLIDE 4

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Introduction

OBIA and Spatial Reasonning vs. segmentation

◮ In VHR satellite imagery, OBIA1 and Spatial Reasoning

widely studied and used

◮ But . . . segmentation pre-processing step:

◮ Lacks of quality and stability across large scenes ◮ Is hardly scalable to those scenes anyway (at least in free

software)

This talk

◮ Tries to address the unscalability ◮ Presents a new framework available as a part of the OTB 2

1Object-Based Image Analysis 2Orfeo ToolBox, Satellite Image open source software developed by CNES

slide-5
SLIDE 5

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Why is it difficult ?

On the image processing side (what this talk is not about)

◮ VHR optical imagery is not a natural imagery (blur, noise,

pan-sharpening . . . )

◮ Scene is VERY large: wide swath with varying conditions

(shadows, specular reflections . . . )

◮ Objects (urban ones) are often small wrt spatial resolution

On the computer science side (what this talk is about)

◮ Data is VERY large: the data does not fit into memory ◮ Segmentation does not cope well with piecewise

computation

◮ Results are hard to store and manipulate efficiently

slide-6
SLIDE 6

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Outline

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

slide-7
SLIDE 7

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Binary and labeled rasters

Principle

◮ Image of integer labels ◮ Unique labels for each segment ◮ Very close to the image

processing perspective

Why it can not meet scalability

◮ Billions of segments: how many bits for unique labels ? ◮ Getting segment pixels or boundaries: intensive image

parsing !

slide-8
SLIDE 8

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Label maps

Principle

◮ Run Length Encoding: from the data compression field ◮ A run (raw of pixels): (start index, length) ◮ A segment: collection of runs ◮ A segmentation: map (dictionary) of (label, segment)

Why it can not meet scalability

◮ Almost images . . . But how to store them ? ◮ Some efficient operations, other are nightmares ◮ Poorer segmentation means poorer memory efficiency

slide-9
SLIDE 9

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Vectors

Principle

◮ Store segments as polygons

Why it can meet scalability

◮ Plenty of file formats from the GIS world ◮ Even Databases (PostGIS . . . ) ◮ Size grows linearly with the amount of segments ◮ No explicit unique labels required ◮ Direct compatibility with GIS world

slide-10
SLIDE 10

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Outline

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

slide-11
SLIDE 11

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

The segmentation algorithms

◮ For now, available algorithms are:

◮ Simple Connected Components with custom criterion ◮ Watershed ◮ Mean-Shift3

◮ Any segmentation algorithm can be fitted in the framework,

if:

◮ It outputs a (partial) lableled raster output ◮ It is written (or encapsulated) as an OTB filter ◮ . . . that’s all: no need for parallel computation compatibility

for instance

We do not want strong constraints on the algorithms!

  • 3D. Comaniciu and P

. Meer, “Mean shift: A robust approach toward feature space analysis,” IEEE Transactions on pattern analysis and machine intelligence, pp. 603–619, 2002.

slide-12
SLIDE 12

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

(New) components in Orfeo ToolBox

◮ OGR encapsulation for large vector file handling ◮ Filters for exact conversion between labeled raster and

vector

◮ Based on GDAL and OGR capabilities ◮ Most segmentation algorithms procude labeled raster

◮ New filter for large scale segmentation

◮ Tile based segmentation and vectorization ◮ Geometries written to disk for each tile ◮ Genericity on segmentation algorithms (templates)

◮ An application to handle both

◮ Standard segmentation with labeled raster output ◮ Large-scale segmentation with vector output

slide-13
SLIDE 13

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Workflow of the Segmentation application in OTB

slide-14
SLIDE 14

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Examples: raster mode (Connected components)

distance<30 or (intensity_p1>400 and intensity_p2 > 400 and distance<50)

  • r(intensity_p1 >1000 and intensity_p2>1000 or (p1b4 <150 and p2b4<150))
slide-15
SLIDE 15

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Examples: raster mode (Watershed)

slide-16
SLIDE 16

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Examples: raster mode (Mean-Shift)

slide-17
SLIDE 17

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Examples: raster mode (Morphological profiles4, WIP)

4Martino Pesaresi and Jon Alti Benediktsson, Member, IEEE: A new

approach for the morphological segmentation of high resolution satellite

  • imagery. IEEE Transactions on geoscience and remote sensing, vol. 39, NO.

2, February 2001, p. 309-320.

slide-18
SLIDE 18

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Examples: vector mode (Mean-Shift)

◮ Larger image : 4096 by 4096 pixels, 4 bands ◮ Requires more than 2 Go of RAM for processing in raster

mode

◮ Requires around 500 Mo for processing in vector mode ◮ In less than 3 minutes ◮ Produces 52799 polygons ◮ An image of 16384 by 16384 pixels would require 48

minutes of processing

◮ But would still require around 500 Mo of RAM

slide-19
SLIDE 19

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Examples: vector mode (Mean-Shift, QGis export)

slide-20
SLIDE 20

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Outline

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

slide-21
SLIDE 21

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

This is great but . . .

◮ Did you notice the tiling effects in the results ? ◮ Do I really need to segment everything in the scene ? ◮ How can I control the size of the output vector file ?

slide-22
SLIDE 22

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Stitching

◮ Simple rule to stitch together polygons split by tiling ◮ We look for neighboring polygons on both side of the split ◮ We merge polygons based on overlap surface ◮ We process horizontal splits first, and then vertical ones

slide-23
SLIDE 23

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Stitching (example)

slide-24
SLIDE 24

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Masking and small objects rejection

We usually do not want small objects

◮ They are almost pure noise ◮ They are hard to classify in OBIA ◮ They grow the size of the output file

We might also want to discard some parts of the image

◮ Based on land-cover or image features ◮ An input mask can be provided ◮ Masked pixels will be discarded during vectorization step

slide-25
SLIDE 25

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Masking and small objects rejection (example)

slide-26
SLIDE 26

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Some other interesting features

Polygon simplification from a given tolerance

◮ Enabled by OGR ◮ Reducing the number of nodes decreases file size

Of course, tile size can be tuned

◮ Can fit a memory limit or an image file format (ex.: fit

JPEG2000 tiles)

OGR options to access more complex files and databases

◮ Seamless options passing to OGR driver ◮ Different writing mode : update layers, create new layers

  • nly, overwrite . . .
slide-27
SLIDE 27

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Outline

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

slide-28
SLIDE 28

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Conclusion

Achieved so far

◮ A tool able to segment full satellite imagery ◮ Generic on the segmentation algorithm side ◮ Already available in OTB 3.14 ◮ With native interoperability with GIS world

What next ?

◮ More clever stitching methods (in progress) ◮ Compute attributes on polygons from image ◮ Learn from and classify large vector files ◮ Large-scale spatial reasoning ?

slide-29
SLIDE 29

Introduction Data representation and conversion Generic framework for large scale segmentation Pre and post processing Conclusion

Thank you!

Orfeo ToolBox webiste http://www.orfeo-toolbox.org Source code http://hg.orfeo-toolbox.org Blog http://blog.orfeo-toolbox.org Wiki http://wiki.orfeo-toolbox.org Users list http://groups.google.fr/group/otb-users Developers list http://groups.google.fr/group/otb-developers