GPU Enabled Machine Learning for Atmospheric Cloud Detection • Byung H. Park, PhD • Ramakrishnan Kannan, PhD • Jeanette Weaver • Kristin Safi, PhD • Sophie Voisin, PhD • David Hughes, PhD Session ID: DCS16152 ORNL is managed by UT-Battelle for the US Department of Energy
Image Quality Team: Cloud Detection • Byung H. Park, PhD Geographic Information Science & Technology Computer Remote • Science / Ramakrishnan Kannan, PhD Sensing HPC • Jeanette Weaver Computer Science Research Group • Kristin Safi, PhD Machine • Sophie Voisin, PhD Nature Inspired Machine Learning Team learning • David Hughes, PhD algorithm development David Hughes 2
PRIMUS Background • PRIMUS (Photogrammetric Registration of Imagery from Manned and Unmanned Systems) • Overhead imaging systems (spaceborne and airborne) can vary substantially in their geopositioning accuracy Satellites • Sponsor sought to establish an automated near real time geocoordinate correction capability. Manned Aircraft • Automated solution uses well-known and cutting- edge photogrammetric, image science, remote sensing, computer vision, machine learning, and high performance computing techniques • On-board processing planned to be in place by 2020. Unmanned Aerial Systems David Hughes 3
PRIMUS: Auto Tie-point Creation with Clouds Visible image with tie-point polygon overlays David Hughes 4
PRIMUS: Auto Tie-point Creation with Clouds This is the problem we are trying to solve If we can solve it… More accurate global localization Speed-up on tie-point generation David Hughes 5
PRIMUS: Auto Tie-point Creation with Clouds Control Imagery Base David Hughes 6
PRIMUS: auto tie-point creation with clouds Control Imagery Base Not-masked and masked overlays David Hughes 7
PRIMUS: Auto Tie-point Creation with Clouds Control Imagery Base Not-masked and masked overlays ~4 km offset David Hughes 8
Goals • Detection must be fast (on the order of < 1 second for a standard image). • Detect clouds regardless of their altitude or geographic location . • Detection must be automatic and require no human interaction. • Detection must be accurate to create a cloud mask to provide to PRIMUS processing. David Hughes 9
Goals • Detection must be fast (on the order of < 1 second for a standard image). • Detect clouds regardless of their altitude or geographic location . • Detection must be automatic and require no human Deep Learning interaction. • Detection must be accurate to create a cloud mask to provide to PRIMUS processing. David Hughes 10
Goals • Detection must be fast (on the order of < 1 second for a GPU enabled computations standard image). Run on a Dell C4130 with four • Tesla K80 cards. This amounts to 8 GPUs dedicated to cloud detect. • • David Hughes 11
Solution for Cloud Detection: A Few Definitions • Machine learning. An algorithm that can learn from and make predictions on data. • Deep learning. Attempt to model features of data using multiple processing layers Hubel, D. H.; Wiesel, T. N. (1968-03-01). "Receptive fields and composed of multiple linear and non-linear functional architecture of monkey striate cortex". The Journal of Physiology. 195 (1): 215 – 243. transformations. • Convolutional neural networks (CNNs). A topology to process visual data, inspired by animal visual cortex (responses to overlapping regions) David Hughes 12
How to pick a Deep Network • CNN models are motivated out of imagenet competition 1 • Alexnet – 2012 – Sequential, additional operations Accuracy beyond standard CNN – Standard operations are convolution, RelU, Maxpool, fully connected • Alexnet for cloud detection – Fine tuned for detecting clouds • VGG Net – 2014 – Sequential like Alexnet but deeper, only standard operations • Google Net – 2014 – Non-sequential • ResNet – 2015 – Combining intermediate stages http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/ https://www.quora.com/What-is-the-state-of-the-art-today-on-ImageNet-classification-In-other-words-has-anybody-beaten-Deep-Residual-Learning 1 hci.stanford.edu/publications/2015/scenegraphs/ imagenet - challenge .pdf David Hughes 13
Cloud Detection Flow Training Set Cloud Model Object Network Segmentation Validation Testing Cloud Model Testing David Hughes 14
Cloud Detection Flow: Model Building Utilizing Caffe Net module Training Set Cloud Model (cuDNN version 5.1) https://developer.nvidia.com/cudnn Object Network Segmentation Model Training time Validation Testing (5000 images) (8 GPUs) Naïve 3 hours alexnet 7 minutes Cloud Model Testing Naïve alexnet with 2 hours cloudless 9 minutes tuning Naïve 31 hours googlenet 54 minutes David Hughes 15
Cloud Detection Flow: Model Building Training Set Cloud Model Object Network Segmentation Validation http://www.alpineeyecare.com/wp-content/uploads/Optometrist-sm.jpg Testing Phoropter Cloud Model Testing David Hughes 16 https://thumbs.dreamstime.com/z/optometrist-diopter-19996868.jpg
Model Building Convolve 1 Training Set (kernel_size, filter) Median Network Convolve 2 Validation (kernel_size, filter) Testing … … David Hughes 17
Model Building Convolve 1 Training Set (kernel_size, filter) Median Network Convolve 2 Validation (kernel_size, filter) Testing … … David Hughes 18
Model Building Convolve 1 Training Set (kernel_size, filter) Median Network Convolve 2 Validation (kernel_size, filter) Testing … … David Hughes 19
Training Set David Hughes 20
Warning in Training a Network The water is misclassified as cloud False color with cloud overlay Visible, 3-band image (Tiling segmentation) David Hughes 21
Are CNNs Effective at Cloud Detection? Cloudless model (~4000 files) 22%: cloud • As a proof of concept, we first 78%: not cloud implemented Cloudless (deep Questionable balance. learning pipeline for orbital satellite data for detecting clouds). • Utilized 7400+ image chips of ORNL cloud detect CNN size 227 x 227 pixels. 1353 pngs completely covered by cloud 1642 pngs with partial cloud cover https://github.com/BradNeuberg/cloudless 4502 pngs with no cloud. Balanced: 50%/50% cloud/no-cloud David Hughes 22
Cloud Detection Flow Training Set Cloud Model Object Network Segmentation Validation Testing Cloud Model Testing David Hughes 23
Cloud Detection Flow Image validation Image Input (angle, location, time) Training Set Cloud Model Object segmentation Object Network Pixel Segmentation Tiling Striding Tile Segmentation Validation Testing 227 pixels 227 pixels Cloud Model Testing David Hughes 24
Cloud Detection Flow Training Set Cloud Model Object Network Segmentation 227 pixels 227 pixels (thousands of these) Validation Testing Cloud Model Testing 60% cloud confidence David Hughes 25
CPU and GPU Testing Implementation GPU CPU Deep neural network (dnn) Caffe using cuDNN version 5.1 installed module of the opencv-contrib from source. package • cuDNN is thread-safe but Caffe is NOT • Does not utilize GPUs inherently thread-safe for testing. • Speed-up from parallelizing • Out of the box, we were not able to boost sub-image testing on multi performance with multiple Caffe threads CPU cores. • OpenMP threads. maximizing GPU(s) utility for testing. • Created separate Caffe instances per thread thereby tuning for a multi-threaded user application. David Hughes 26
GPU/CPU Timings Comparison for Testing We parallelize the entire inferences using multiple OpenMP threads over multiple GPUs (sharing of a single GPU by multiple threads are also possible). 1 GPU-thread 3.9 seconds/image 2 GPU-threads 1.8 seconds/image 4 GPU-threads 1.3 seconds/image 8 GPU-threads 0.64 seconds/image David Hughes 27
GPU-enabled Timings for Testing (with Batching) • 2x-3x speed-up to process each image utilizing batching. • Further time decreases expected with higher resolution image striding. David Hughes 28
Caveats • Only using visible region data (Red, Green, Blue or Panchromatic) • Only using near-nadir viewing data. • Not looking at clouds over ice or snow. • Not looking at clouds near dusk/dawn/nighttime. • Caveats are based on currently available data that we have tested against. David Hughes 29
Cloud Detection Imagery Cloud David Hughes 30
Cloud Detection Imagery Cloud Cloud Cloud Tiling Striding Tile (Step 50) Visible, 3-band Image David Hughes 31
Cloud Detection Imagery Tiling Striding Tile (Step 50) Visible, 3-band image David Hughes 32
Cloud Detection Imagery Tiling Striding Tile (Step 50) Visible, 3-band image David Hughes 33
Conclusion and Next Steps • Novel, previously unavailable implementation of GPU-enabled inference. • Cloud confidence heat map creation (thousands of image chips analyzed) made possible by GPU-enabled calculation. • Extensible algorithm for cloud and other objects of interest Flooding (unexpected (cloud shadow and water) Cloud shadow occurrence of water) • Object segmentation variation. • Other non-visible modalities • Speed-up David Hughes 34
Recommend
More recommend
