GPU Enabled Machine Learning for Atmospheric Cloud Detection - - PowerPoint PPT Presentation

▶
gpu enabled machine
SMART_READER_LITE
LIVE PREVIEW

GPU Enabled Machine Learning for Atmospheric Cloud Detection - - PowerPoint PPT Presentation

Nov 22, 2023 120 likes •465 views

GPU Enabled Machine Learning for Atmospheric Cloud Detection Byung H. Park, PhD Ramakrishnan Kannan, PhD Jeanette Weaver Kristin Safi, PhD Sophie Voisin, PhD David Hughes, PhD Session ID: DCS16152 ORNL is

slide-1
SLIDE 1

ORNL is managed by UT-Battelle for the US Department of Energy

  • Byung H. Park, PhD
  • Ramakrishnan Kannan, PhD
  • Jeanette Weaver
  • Kristin Safi, PhD
  • Sophie Voisin, PhD
  • David Hughes, PhD

Session ID: DCS16152

GPU Enabled Machine Learning for Atmospheric Cloud Detection

slide-2
SLIDE 2

2 David Hughes

Image Quality Team: Cloud Detection

Remote Sensing Machine learning algorithm development Computer Science / HPC

Geographic Information Science & Technology Computer Science Research Group Nature Inspired Machine Learning Team

  • Byung H. Park, PhD
  • Ramakrishnan Kannan, PhD
  • Jeanette Weaver
  • Kristin Safi, PhD
  • Sophie Voisin, PhD
  • David Hughes, PhD
slide-3
SLIDE 3

3 David Hughes

PRIMUS Background

  • PRIMUS (Photogrammetric Registration of Imagery

from Manned and Unmanned Systems)

  • Overhead imaging systems (spaceborne and

airborne) can vary substantially in their geopositioning accuracy

  • Sponsor sought to establish an automated near real

time geocoordinate correction capability.

  • Automated solution uses well-known and cutting-

edge photogrammetric, image science, remote sensing, computer vision, machine learning, and high performance computing techniques

  • On-board processing planned to be in place by 2020.

Satellites Manned Aircraft Unmanned Aerial Systems

slide-4
SLIDE 4

4 David Hughes

PRIMUS: Auto Tie-point Creation with Clouds

Visible image with tie-point polygon overlays

slide-5
SLIDE 5

5 David Hughes

PRIMUS: Auto Tie-point Creation with Clouds

This is the problem we are trying to solve If we can solve it… More accurate global localization Speed-up on tie-point generation

slide-6
SLIDE 6

6 David Hughes

PRIMUS: Auto Tie-point Creation with Clouds

Control Imagery Base

slide-7
SLIDE 7

7 David Hughes

PRIMUS: auto tie-point creation with clouds

Control Imagery Base Not-masked and masked overlays

slide-8
SLIDE 8

8 David Hughes

PRIMUS: Auto Tie-point Creation with Clouds

Control Imagery Base Not-masked and masked overlays ~4 km offset

slide-9
SLIDE 9

9 David Hughes

Goals

  • Detection must be fast (on the
  • rder of < 1 second for a

standard image).

  • Detect clouds regardless of

their altitude or geographic location.

  • Detection must be automatic

and require no human interaction.

  • Detection must be accurate to

create a cloud mask to provide to PRIMUS processing.

slide-10
SLIDE 10

10 David Hughes

Goals

  • Detection must be fast (on the
  • rder of < 1 second for a

standard image).

  • Detect clouds regardless of

their altitude or geographic location.

  • Detection must be automatic

and require no human interaction.

  • Detection must be accurate to

create a cloud mask to provide to PRIMUS processing.

Deep Learning

slide-11
SLIDE 11

11 David Hughes

Goals

  • Detection must be fast (on the
  • rder of < 1 second for a

standard image).

  • GPU enabled computations

Run on a Dell C4130 with four Tesla K80 cards. This amounts to 8 GPUs dedicated to cloud detect.

slide-12
SLIDE 12

12 David Hughes

  • Machine learning. An algorithm that can

learn from and make predictions on data.

  • Deep learning. Attempt to model features
  • f data using multiple processing layers

composed of multiple linear and non-linear transformations.

  • Convolutional neural networks (CNNs). A

topology to process visual data, inspired by animal visual cortex (responses to

  • verlapping regions)

Solution for Cloud Detection: A Few Definitions

Hubel, D. H.; Wiesel, T. N. (1968-03-01). "Receptive fields and functional architecture of monkey striate cortex". The Journal of

  • Physiology. 195 (1): 215–243.
slide-13
SLIDE 13

13 David Hughes

How to pick a Deep Network

  • CNN models are motivated out of imagenet

competition1

  • Alexnet – 2012 – Sequential, additional operations

beyond standard CNN – Standard operations are convolution, RelU, Maxpool, fully connected

  • Alexnet for cloud detection – Fine tuned for detecting

clouds

  • VGG Net – 2014 – Sequential like Alexnet but deeper,
  • nly standard operations
  • Google Net – 2014 – Non-sequential
  • ResNet – 2015 – Combining intermediate stages

1hci.stanford.edu/publications/2015/scenegraphs/imagenet-challenge.pdf

Accuracy http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/

https://www.quora.com/What-is-the-state-of-the-art-today-on-ImageNet-classification-In-other-words-has-anybody-beaten-Deep-Residual-Learning

slide-14
SLIDE 14

14 David Hughes

Cloud Detection Flow

Training Set Validation Testing Network Object Segmentation Testing Cloud Model Cloud Model

slide-15
SLIDE 15

15 David Hughes

Cloud Detection Flow: Model Building

Model

(5000 images)

Training time (8 GPUs)

Naïve alexnet 3 hours 7 minutes Naïve alexnet with cloudless tuning 2 hours 9 minutes Naïve googlenet 31 hours 54 minutes

Utilizing Caffe Net module (cuDNN version 5.1)

https://developer.nvidia.com/cudnn

Training Set Validation Testing Network Object Segmentation Testing Cloud Model Cloud Model

slide-16
SLIDE 16

16 David Hughes

Cloud Detection Flow: Model Building

http://www.alpineeyecare.com/wp-content/uploads/Optometrist-sm.jpg https://thumbs.dreamstime.com/z/optometrist-diopter-19996868.jpg

Phoropter

Training Set Validation Testing Network Object Segmentation Testing Cloud Model Cloud Model

slide-17
SLIDE 17

17 David Hughes

Model Building

Training Set Validation Testing Network Convolve1

(kernel_size, filter)

Median Convolve2

(kernel_size, filter)

… …

slide-18
SLIDE 18

18 David Hughes

Model Building

Training Set Validation Testing Network Convolve1

(kernel_size, filter)

Median Convolve2

(kernel_size, filter)

… …

slide-19
SLIDE 19

19 David Hughes

Model Building

Training Set Convolve1

(kernel_size, filter)

Validation Testing Median Convolve2

(kernel_size, filter)

… …

Network

slide-20
SLIDE 20

20 David Hughes

Training Set

slide-21
SLIDE 21

21 David Hughes

Warning in Training a Network

Visible, 3-band image False color with cloud overlay (Tiling segmentation) The water is misclassified as cloud

slide-22
SLIDE 22

22 David Hughes

Are CNNs Effective at Cloud Detection?

  • As a proof of concept, we first

implemented Cloudless (deep learning pipeline for orbital satellite data for detecting clouds).

  • Utilized 7400+ image chips of

size 227 x 227 pixels. Cloudless model (~4000 files) 22%: cloud 78%: not cloud Questionable balance. ORNL cloud detect CNN 1353 pngs completely covered by cloud 1642 pngs with partial cloud cover 4502 pngs with no cloud. Balanced: 50%/50% cloud/no-cloud

https://github.com/BradNeuberg/cloudless

slide-23
SLIDE 23

23 David Hughes

Cloud Detection Flow

Training Set Validation Testing Network Object Segmentation Testing Cloud Model Cloud Model

slide-24
SLIDE 24

24 David Hughes

Cloud Detection Flow

Object segmentation

Image Input Image validation (angle, location, time) Tiling Striding Tile

Pixel Segmentation

227 pixels 227 pixels

Training Set Validation Testing Network Object Segmentation Testing Cloud Model Cloud Model

slide-25
SLIDE 25

25 David Hughes

Cloud Detection Flow

227 pixels 227 pixels

60% cloud confidence Training Set Validation Testing Network Object Segmentation Testing Cloud Model Cloud Model (thousands of these)

slide-26
SLIDE 26

26 David Hughes

CPU and GPU Testing Implementation

Caffe using cuDNN version 5.1 installed from source.

  • cuDNN is thread-safe but Caffe is NOT

inherently thread-safe for testing.

  • Out of the box, we were not able to boost

performance with multiple Caffe threads maximizing GPU(s) utility for testing.

  • Created separate Caffe instances per

thread thereby tuning for a multi-threaded user application.

CPU GPU

Deep neural network (dnn) module of the opencv-contrib package

  • Does not utilize GPUs
  • Speed-up from parallelizing

sub-image testing on multi CPU cores.

  • OpenMP threads.
slide-27
SLIDE 27

27 David Hughes

GPU/CPU Timings Comparison for Testing

We parallelize the entire inferences using multiple OpenMP threads over multiple GPUs (sharing

  • f a

single GPU by multiple threads are also possible). 1 GPU-thread  3.9 seconds/image 2 GPU-threads  1.8 seconds/image 4 GPU-threads  1.3 seconds/image 8 GPU-threads  0.64 seconds/image

slide-28
SLIDE 28

28 David Hughes

GPU-enabled Timings for Testing (with Batching)

  • 2x-3x speed-up to process each

image utilizing batching.

  • Further time decreases expected

with higher resolution image striding.

slide-29
SLIDE 29

29 David Hughes

Caveats

  • Only using visible region data

(Red, Green, Blue or Panchromatic)

  • Only using near-nadir viewing

data.

  • Not looking at clouds over ice or

snow.

  • Not looking at clouds near

dusk/dawn/nighttime.

  • Caveats are based on currently

available data that we have tested against.

slide-30
SLIDE 30

30 David Hughes

Cloud Detection Imagery

Cloud

slide-31
SLIDE 31

31 David Hughes

Cloud Detection Imagery

Visible, 3-band Image Tiling Striding Tile (Step 50)

Cloud Cloud Cloud

slide-32
SLIDE 32

32 David Hughes

Visible, 3-band image Tiling Striding Tile (Step 50)

Cloud Detection Imagery

slide-33
SLIDE 33

33 David Hughes

Visible, 3-band image Tiling Striding Tile (Step 50)

Cloud Detection Imagery

slide-34
SLIDE 34

34 David Hughes

Conclusion and Next Steps

  • Novel, previously unavailable

implementation of GPU-enabled inference.

  • Cloud confidence heat map

creation (thousands of image chips analyzed) made possible by GPU-enabled calculation.

  • Extensible algorithm for cloud

and other objects of interest (cloud shadow and water)

  • Object segmentation variation.
  • Other non-visible modalities
  • Speed-up

Flooding (unexpected

  • ccurrence of water)

Cloud shadow