Intrinsic Quality Analysis of Binary Partition Trees Jimmy Francky - PowerPoint PPT Presentation

Introduction Intrinsic evaluation of the quality of a BPT Experimental study Conclusions and Perspectives Intrinsic Quality Analysis of Binary Partition Trees Jimmy Francky Randrianasoa 1 Camille Kurtz 2 Pierre Gançarski 1 Éric Desjardin 3 Nicolas Passat 3 1 Université de Strasbourg, ICube, France 2 Université Paris-Descartes, LIPADE, France 3 Université de Reims Champagne-Ardenne, CReSTIC, France ICPRAI 2018 Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 1

Introduction Intrinsic evaluation of the quality of a BPT Experimental study Conclusions and Perspectives Plan Introduction 1 Intrinsic evaluation of the quality of a BPT 2 Experimental study 3 Conclusions and Perspectives 4 Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 2

Introduction Intrinsic evaluation of the quality of a BPT Context Experimental study Problematic and purpose Conclusions and Perspectives Plan Introduction 1 Intrinsic evaluation of the quality of a BPT 2 Experimental study 3 Conclusions and Perspectives 4 Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 3

Introduction Intrinsic evaluation of the quality of a BPT Context Experimental study Problematic and purpose Conclusions and Perspectives Hierarchical image representation Hierarchical segmentation Better consideration of complex and heterogeneous objects Construction of hierarchical models as image representation (e.g., min-tree, max-tree, inclusion-tree, BPT , ...) Application of a segmentation method on the hierarchical model (e.g., cut, node selection, ...) Various segmentations for different level Figure: Different level scales scales Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 4

Introduction Intrinsic evaluation of the quality of a BPT Context Experimental study Problematic and purpose Conclusions and Perspectives Hierarchical image representation Binary Partition Tree (BPT) [Salembier and Garrido, 2000] A hierarchical representation of the regions contained in an image Leaves: elementary regions defined by an initial partition L (pixels, pre-segmentation) Nodes: fusion of two neighbouring regions according to a metric of similarity W Root: node representing the image support The BPT is a hierarchical structure, frequently involved in image segmentation procedures. Figure: Creation based on a bottom-up algorithm Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 5

Introduction Intrinsic evaluation of the quality of a BPT Context Experimental study Problematic and purpose Conclusions and Perspectives Metric of similarity W involved during the BPT construction Figure: One image ⇒ various BPTs according to the metrics W used. Prior information Defined by the user (expert) Homogeneity criterion (radiometric, geometric,. . . ) Metric W : valuation function defining the similarity between two neighbouring regions ⇒ The metric W guides the node merging process. Neighbouring regions Feature Metric Radiometric Intensity Geometric Elongation Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 6

Introduction Intrinsic evaluation of the quality of a BPT Context Experimental study Problematic and purpose Conclusions and Perspectives Image segmentation from a BPT Image segmentation by cutting a BPT Extraction of homogeneous relevant areas ( partitioning an image) BPT cut Image Borders Partition Figure: Example of a segmentation result by partitioning a remote sensing image. Various cutting methods Horizontal cut Optimal cut ... Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 7

Introduction Intrinsic evaluation of the quality of a BPT Context Experimental study Problematic and purpose Conclusions and Perspectives Impact of the quality of a hierarchical image representation for further image segmentation For the intrinsic image models (e.g., component trees) The quality of the result depends on the segmentation method (e.g., cut on the tree) For the mixed hierarchical structures (e.g., BPT) The quality of the result does not only depend on the segmentation method (e.g., cut on the tree) The quality of the result depends also on the quality of the BPT ⇒ A good segmentation method should be applied on a good BPT. Quality of a BPT Capacity of a BPT to provide relevant segmentation results for the user Depending on the construction of the BPT according to a given metric W Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 8

Introduction Intrinsic evaluation of the quality of a BPT Context Experimental study Problematic and purpose Conclusions and Perspectives Problematic and purpose Problematic How to evaluate the capacity of a BPT to provide relevant segmentation results? ? ? ? Is the BPT "good" ? ? ? How to evaluate ? cut cut result result cut cut result result cut cut result cut result result How to chose a good metric / feature to build a BPT ? Purpose Intrinsic evaluation of the quality of a BPT or, equivalently, its construction process ⇒ help the user to choose a right BPT (i.e., BPT evaluation) but not to use it the right way (i.e., not a segmentation evaluation) Evaluation based on comparisons to a set of segments of reference (a.k.a. ground-truth examples) Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 9

Introduction Related works Intrinsic evaluation of the quality of a BPT New approach for the evaluation of the quality of a BPT Experimental study Intrinsic analysis of the quality of a BPT Conclusions and Perspectives Plan Introduction 1 Intrinsic evaluation of the quality of a BPT 2 Experimental study 3 Conclusions and Perspectives 4 Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 10

Introduction Related works Intrinsic evaluation of the quality of a BPT New approach for the evaluation of the quality of a BPT Experimental study Intrinsic analysis of the quality of a BPT Conclusions and Perspectives Related works on the evaluation of the quality of a BPT Segmentation quality evaluation Unsupervised approaches : computation of properties (homogeneity, etc.) on the segments [Troya-Galvis et al, 2015] Supervised approaches : comparison of regions to some segments of reference [Vojodi and Moghadam, 2012] [Pont-Tuset and Marques, 2016] Metrics based on the overlapping of a region N and a segment of reference G : Jaccard index [Jaccard, 1912] Dice Coefficient (a.k.a F-mesure) [Dice, 1945] Other approaches: object oriented metrics and border oriented metrics Related works on the evaluation of hierarchical models [Pont-Tuset et al, 2012] Selecting, in the tree, a set of segments matching an ideal partition that is forced to be in the hierarchy [Perret et al., 2017] Evaluation of hierarchical watersheds [Randrianasoa et al, 2017] Extrinsic quality evaluation of binary partition trees Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 11

Introduction Related works Intrinsic evaluation of the quality of a BPT New approach for the evaluation of the quality of a BPT Experimental study Intrinsic analysis of the quality of a BPT Conclusions and Perspectives New approach for the evaluation of the quality of a BPT Expert Problem of Difficulty of the adaptation of the classical uncertain borders methods Several possible results from one BPT Quality of a BPT depending on the Semantic labels Image expectation of the user Built area Forest area Choice of the segments of reference Herbaceous area Shadow Ground-truth map Supervised approach for the evaluation of the quality of a BPT Intrinsic analysis : evaluation from the internal coherence of the hierarchical structure Evaluate the quality of a BPT (or equivalently its construction process ) Reliable clues for such quality analysis can be obtained directly investigating the BPT structure with respect to partial ground-truth (GT) examples . Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 12

Introduction Related works Intrinsic evaluation of the quality of a BPT New approach for the evaluation of the quality of a BPT Experimental study Intrinsic analysis of the quality of a BPT Conclusions and Perspectives Steps of the intrinsic BPT quality analysis relying on GT examples (a) G 1 (b) (c) G 9 (d) (e) G 11 (f) (g) G 12 (h) Figure: Examples of segments of reference G (partial ground-truth (GT) examples) Direct observation of the intern structure of the BPT Choice of the partial ground-truth examples by the user Step 1 : extraction of a subtree T G (intersecting the segment of reference G ) Step 2 : verification of the relevance of the intrinsic BPT quality analysis Step 3 : Intrinsic analysis: directly investigate the BPT structure for the quality evaluation Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 13

Introduction Related works Intrinsic evaluation of the quality of a BPT New approach for the evaluation of the quality of a BPT Experimental study Intrinsic analysis of the quality of a BPT Conclusions and Perspectives Step 1: extraction of a subtree “example-based” Families of nodes Intersecting the segment of reference G Not intersecting the segment of reference G Tianatahina Jimmy Francky Randrianasoa ICPRAI 2018 - page 14