Satellite Imagery Semantic Segmentation Razieh Kaviani Baghbaderani, - PowerPoint PPT Presentation

Incorporating Spectral Unmixing in Satellite Imagery Semantic Segmentation Razieh Kaviani Baghbaderani, Hairong Qi University of Tennessee, Department of EECS

Motivation: ▪ Deep learning-based networks need a large dataset for training due to a huge number of hyper-parameters. Encoder Decoder ▪ Challenges in remote sensing area: o Difficulty in obtaining a large number of labeled training samples due to the high- resolution and large coverage of satellite images. o The existence of mixed pixels that would result in mixed spectral readings due to environmental interference as well as the large footprint of the sensing device.

Main Idea: ▪ Employing the spectral unmixing method as a feature extraction step to entangle the intertwined reflectances of the given images and obtain the advanced features as extra domain information. ▪ Decompose a mixture into constituent materials (endmembers, A) and their proportions (abundance, S). ▪ Physical constraints (nonnegativity & sum- to-one) ▪ Unsupervised (both A and S are unknown) vs. Supervised (A is known)

Approach: MVC-NMF FCLS

Dataset: ▪ Dataset was obtained from Kaggle Competition: “ Dstl Satellite Imagery Feature Detection”  Imbalanced data Distributions of target classes Standing water Waterways Crops Trees Track Structure Building 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00%

Experimental Results: Test image: ▪ The probability of pixels belonging to each class

Experimental Results: ▪ The segmentation accuracy (without any post processing) for different classes in terms of Intersection Over Union (IOU%) 70 60 50 40 30 20 10 0 Building Structure Track Trees Crops Avg. Baseline [Pan, RGB, MSI] Unmixing-based [Pan, RGB, MSI] Baseline [Pan, RGB, MSI, 4Ratios] Unmixing-based [Pan, RGB, MSI, 4Ratios]

Conclusion: ▪ The abundance maps as extra information along with spectral bands enhance the segmentation accuracy and increase the confidence of network in predicting the label for each pixel. ▪ Adding abundance maps to improve the segmentation accuracy is more effective when the training samples are limited