Hierarchy Construction Schemes within the Scale Set Framework - PowerPoint PPT Presentation

Hierarchy Construction Schemes within the Scale Set Framework Jean-Hugues PRUVOT, Luc BRUN GreyC Laboratory, Image Team CNRS UMR 6072 ENSICAEN 6th IAPR -TC-15 Workshop on Graph-based Representations in Pattern Recognition J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 1 / 22

Outline Outline Introduction 1 2 The Scale Set Framework The Causality principle Optimal Cuts Merging Heuristics 3 Sequential Merging Parallel Merging Conclusion and Outlooks 4 J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 2 / 22

Introduction Introduction 2 approaches for Segmentation Segmentation ⇒ Horowitz[1976] with a predicate P ◮ split/merge while an homogeneity criteria ◮ only a local criteria The use of energy minimisation scheme within the region based segmentation framework ◮ Level-Set, Bayesian, Min-cut / N-Cut, Minimum Description Length ◮ allows to define criteria which should be globally optimised over a partition ◮ allows an objective evaluation the segmentations ◮ Level-Set ⇒ minimisation for one scale parameter λ J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 3 / 22

The Scale Set Framework The Causality principle The Scale Set Framework Level Set / N-Cut / MDL ⇒ local minimum in a full search space Principle minimize an energy partition E ( P ) : ( E ( P ) is supposed to be an Affine Separable Energy (ASE) ) ◮ Energy for each region R i (weighted sum of 2 terms) : � E ( R ) = D ( R i ) + λ C ( R i ) R i ∈ R ◮ D ( R i ) : internal region energy (fit to data) ◮ C ( R i ) : complexity energy (regularization term) ◮ λ : scale parameter J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 4 / 22

The Scale Set Framework The Causality principle The Scale Set Framework Guigues approach ◮ global minimum in a narrow search space ◮ bottom-up approach ◮ provides the optimal partition for each value of a scale parameter λ ∈ R + Principle minimize an energy partition E ( P ) : ( E ( P ) is supposed to be an Affine Separable Energy (ASE) ) ◮ Energy for each region R i (weighted sum of 2 terms) : � E ( R ) = D ( R i ) + λ C ( R i ) R i ∈ R ◮ D ( R i ) : internal region energy (fit to data) ◮ C ( R i ) : complexity energy (regularization term) ◮ λ : scale parameter J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 4 / 22

The Scale Set Framework The Causality principle The Scale Set Framework Guigues approach ◮ global minimum in a narrow search space ◮ bottom-up approach ◮ provides the optimal partition for each value of a scale parameter λ ∈ R + Principle minimize an energy partition E ( P ) : ( E ( P ) is supposed to be an Affine Separable Energy (ASE) ) ◮ Energy for each region R i (weighted sum of 2 terms) : � E ( R ) = D ( R i ) + λ C ( R i ) R i ∈ R ◮ D ( R i ) : internal region energy (fit to data) ◮ C ( R i ) : complexity energy (regularization term) ◮ λ : scale parameter J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 4 / 22

The Scale Set Framework The Causality principle The Scale Set Framework Guigues approach ◮ global minimum in a narrow search space ◮ bottom-up approach ◮ provides the optimal partition for each value of a scale parameter λ ∈ R + Principle minimize an energy partition E ( P ) : ( E ( P ) is supposed to be an Affine Separable Energy (ASE) ) ◮ Energy for each region R i (weighted sum of 2 terms) : � E ( R ) = D ( R i ) + λ C ( R i ) R i ∈ R ◮ D ( R i ) : internal region energy (fit to data) ◮ C ( R i ) : complexity energy (regularization term) ◮ λ : scale parameter J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 4 / 22

The Scale Set Framework The Causality principle used energy Energy E λ ( P ) = λ C ( P ) + D ( P ) affine energy D : internal energy (fit to data) : ◮ squared error : i ∈ R || c i − µ R || 2 SE ( R ) = � ◮ ⇒ minimal if each region is a pixel C : energy of complexity (regularisation term) : ◮ total length of the boundaries ◮ ⇒ low if the partition is composed of few regions J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 5 / 22

The Scale Set Framework The Causality principle The Scale Set Framework Causality principle introduced by Witkin in 1984 ◮ coherent structures are present at different scales in an image ∀ ( λ 1 , λ 2 ) ∈ R + 2 with λ 2 ≤ λ 1 ⇒ P λ 1 can be deduced from P λ 2 by regions merging J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 6 / 22

The Scale Set Framework The Causality principle The Scale Set Framework Causality principle Conditions on energy E λ ( P ) = λ C ( P ) + D ( P ) ◮ P λ ( I ) : partition which minimize E λ ( I ) ◮ if P λ is causal , H = { P λ ( I ) , λ ∈ R + } is a hierarchy. Guigues ⇒ if C is sub-additive then P is causal ◮ C sub-additive ⇐⇒ C ( R 1 ∪ R 2 ) ≤ C R 1 + C R 2 ◮ natural condition in segmentation task ( MDL ⇒ less parameters to describe the union of 2 regions than 2 regions) J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 7 / 22

The Scale Set Framework The Causality principle Construction of the initial hierarchy2 scale-climbing start from an initial segmentation J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 8 / 22

The Scale Set Framework The Causality principle Construction of the initial hierarchy2 scale-climbing built a region adjacency graph (RAG) J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 8 / 22

The Scale Set Framework The Causality principle Construction of the initial hierarchy2 scale-climbing For each region compute ◮ the internal energy D ( R i ) ◮ the complexity energy C ( R i ) J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 8 / 22

The Scale Set Framework The Causality principle Construction of the initial hierarchy2 scale-climbing Compute, for any couple of adjacent regions, the scale of appearance λ app . λ app depicts the minimum value, from which , merging those regions, contributes to less increase the global energy defined on the partition. J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 8 / 22

The Scale Set Framework The Causality principle Construction of the initial hierarchy2 scale-climbing E A ( λ ) + E B ( λ ) = λ ( C A + C B ) + ( D A + D B ) � λ app ( A ∪ B ) = D A + D B − D A ∪ B E A ∪ B ( λ ) = λ C A ∪ B + D A ∪ B C B + C A − C A ∪ B iterating this process we get a serial of λ app providing a set of optimal cut whitin H energy of the optimal cuts within this global hierarchy is then depicted by a concave piecewise function J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 8 / 22

The Scale Set Framework Optimal Cuts Construction of the initial hierarchy Cuts For a given λ , we retrieve the optimal partition within H J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 9 / 22

The Scale Set Framework Optimal Cuts Construction of the initial hierarchy Cuts For a given λ , we retrieve the optimal partition within H J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 9 / 22

The Scale Set Framework Optimal Cuts Construction of the initial hierarchy Cuts For a given λ , we retrieve the optimal partition within H J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 9 / 22

The Scale Set Framework Optimal Cuts Construction of the initial hierarchy Cuts For a given λ , we retrieve the optimal partition within H J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 9 / 22

The Scale Set Framework Optimal Cuts Construction of the initial hierarchy Cuts For a given λ , we retrieve the optimal partition within H J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 9 / 22

The Scale Set Framework Optimal Cuts Construction of the initial hierarchy Cuts For a given λ , we retrieve the optimal partition within H J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 9 / 22

The Scale Set Framework Optimal Cuts Optimal Cut Briefly provide all solutions for any λ the given solutions ◮ are optimal within the hierarchy corresponding to a narrow search space ◮ Partitions remains stable on whole intervals J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 10 / 22

The Scale Set Framework Optimal Cuts Optimal Cut Briefly provide all solutions for any λ the given solutions ◮ are optimal within the hierarchy corresponding to a narrow search space ◮ Partitions remains stable on whole intervals J.H. PRUVOT, L.BRUN (GreyC) Scale Set representation for image segmentation GbR2007 - 2007-6-11 10 / 22