Investigating the Use of GPU-Accelerated Nodes for SAR Image - - PowerPoint PPT Presentation

▶
investigating the use of gpu accelerated nodes for sar
SMART_READER_LITE
LIVE PREVIEW

Investigating the Use of GPU-Accelerated Nodes for SAR Image - - PowerPoint PPT Presentation

Mar 30, 2024 141 likes •416 views

Investigating the Use of GPU-Accelerated Nodes for SAR Image Formation Timothy D. R. Hartley 1,2 , Ahmed R. Fasih 2 , Charles A. Berdanier 3 , Fusun Ozguner 2 , Umit V. Catalyurek 1,2 1 Department of Biomedical Informatics, 2 Department of

slide-1
SLIDE 1

1

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Investigating the Use of GPU-Accelerated Nodes for SAR Image Formation

Timothy D. R. Hartley1,2, Ahmed R. Fasih2, Charles A. Berdanier3, Fusun Ozguner2, Umit V. Catalyurek1,2

1Department of Biomedical Informatics, 2Department of Electrical and Computer Engineering,

The Ohio State University, Columbus, OH, USA.

3Air Force Research Laboratory,

Wright-Patterson Air Force Base, OH 45433

slide-2
SLIDE 2

2

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Outline

  • Motivation for using GPU clusters
  • SAR overview
  • Software for programming GPU clusters
  • Backprojection implementation
  • Experimental results
  • Conclusions and future work
slide-3
SLIDE 3

3

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Application Motivation

  • SAR image formation is time-consuming
  • Forming 2kx2k image with a small input set takes
  • ver 60 seconds on one CPU core
  • SAR image formation is highly parallel
  • Each output pixel is independently computed
  • Input data can be partitioned also
  • SAR datasets are often large
slide-4
SLIDE 4

4

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Hardware Motivation

Source: Nvidia

slide-5
SLIDE 5

5

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

SAR overview

  • Spotlight-mode

Synthetic Aperture Radar (SAR) aims a radar beam at 'scene center'

  • Records radio pulse

reflections from multiple azimuth angles (1-d line projections)

slide-6
SLIDE 6

6

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

1-d Line Projections

slide-7
SLIDE 7

7

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Image Formation

  • For each input, loop over the output pixels
  • For each output pixel, determine the

contribution of the input line projection

slide-8
SLIDE 8

8

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

  • Application is decomposed into a task-graph
  • T

ask graph performs computation

  • Individual tasks perform single function
  • T

asks are independent, with well-defined interfaces

  • Higher-level programming abstraction
  • DataCutter
  • Coarse-grained filter-stream framework
  • OSU/Maryland-bred component-based framework
  • Third-generation runtime uses MPI for high-

bandwidth network support

Component-based Programming

slide-9
SLIDE 9

9

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

SAR Imaging Pipeline

  • Imaging pipeline composed of three coarse-

grained filters connected by data streams

  • 'Form Partial Image' filter is the time-

consuming task = perform on GPU

slide-10
SLIDE 10

10

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Partitioning Input/Output

  • T
  • map to a GPU cluster for even faster

processing, we need to partition work

  • Partition Input (PI)
  • Simple to partition
  • Input dataset consists of vectors of range profiles
  • Partition Output (PO)
  • Simple to partition
  • Output dataset consists of image pixels
slide-11
SLIDE 11

11

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Partitioning Input

  • Partition input into equal pieces based on

number of 'Form Partial Image' filters

  • Send input partitions to downstream filters
  • Image formation filters output whole range of

image pixels with partial results

  • Aggregate final image by accumulation partial

results

slide-12
SLIDE 12

12

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Partitioning Output

  • Partition output from 'Form Partial Image'

filters

  • Broadcast input from 'Read Input Data' filter
  • Each image formation filter only outputs

portion of whole output image

  • Aggregate final image by simple memcpy
slide-13
SLIDE 13

13

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Combining DataCutter and CUDA

  • DataCutter uses a simple API
  • init(), process(), finish() functions
  • process() function usually implemented as loop
  • Read in data from upstream
  • Process data somehow
  • Write data to output stream
  • CPU implementation inline in process()

function

  • CUDA implementation a function call
  • gpu_backproj() (for example)
  • DataCutter provides access to DCBuffer memory

area with pointers – pass to CUDA function

slide-14
SLIDE 14

14

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

GPU Filter Pseudocode

1 process() { 2 // ... setup constants, read global values from runtime ... 3 DCBuffer * buffer; 4 while((buffer = read(“in”) != NULL) { 5 // ... get data from buffer about data size ... 6 7 // ... get ptr and increment extract index ... 8 phd.real = (float *) buffer->getPtrExtract(); 9 buffer->incrementExtractPointer( ... ); 10 11 // ... prealloc. outgoing buffer and get ptrs ... 12 13 gpu_backproj( ... ); 14 } 15 }

slide-15
SLIDE 15

15

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

CUDA Backprojection

  • Fairly straightforward triple-loop computation
  • Threads calculate one pixel's values based on all

input projections

  • Thread blocks are rectangular sub-images
  • Interesting wrinkles
  • Line projections and sensor location information

can be stored as textures

  • Leverage texture cache, which is faster than

global memory

  • Leverage linear interpolation
  • Required because seldom will pixel centers fall directly
  • n a line projection sample
  • 32 KB shared memory used to store sub-images
slide-16
SLIDE 16

16

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Experiments: System

  • Perform tests on Ohio Supercomputer

Center's BALE cluster

  • BALE nodes
  • 2x AMD dual-core Athlon CPUs
  • 2x NVIDIA Quadro 5600 GPUs
  • 1.5 GB memory
  • G80-based (CUDA compute capability 1.0)
  • 4 GB main memory
  • Infiniband NICs
slide-17
SLIDE 17

17

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Experiments: Input and Output

  • GOTCHA input dataset
  • Air Force Research Lab's Sensor Data Management

System

  • SAR phase history data collected with a 640 MHz

bandwidth

  • Multiple elevation angles (we only make use of one

in our experiments)

  • Eleven azimuth angles
  • Parking lot with various cars and construction

vehicles

  • Three output image sizes (square)
  • 512 – SM, 2048 – MED, 4096 - LRG
slide-18
SLIDE 18

18

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

GOTCHA Images

slide-19
SLIDE 19

19

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Experiments: Implementations

  • C/MPI implementation
  • Very simple multi-process version
  • No SIMD, other optimizations
  • DataCutter/C++ implementation
  • Use kernel from C/MPI version
  • Multithreaded, distributed
  • C/CUDA implementation
  • Single GPU
  • DataCutter/CUDA implementation
  • Multithreaded, distributed, multi-GPU
slide-20
SLIDE 20

20

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

CPU Scalability Results

  • Experiments run with one degree of input
  • DataCutter scales slightly better than MPI
  • Due to better overlap between computation and

communication

slide-21
SLIDE 21

21

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Single GPU Results

  • One degree of input
  • DataCutter

introduces small

  • verhead
  • Due to process

invocation, higher- level paradigm, etc.

  • GPU execution

times scale more than 2x better than linearly with number of pixels

slide-22
SLIDE 22

22

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

CPU/GPU Scalability

  • One degree of input, 4Kx4K (LRG) image size
  • Begin to see divergence on GPUs for input

and output partitioning

slide-23
SLIDE 23

23

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Large GPU Results: DataCutter

  • 11 degrees of data (largest dataset)
  • Good scalability up to 8 GPUs
  • Much better scalability with output partition
slide-24
SLIDE 24

24

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Conclusions and Future Work

  • DataCutter is appropriate for coarse-grained

GPU cluster applications

  • MPI-based runtime uses high-speed interconnects;

ready for HPC applications

  • Encapsulated GPU filter code means easy

application development, usage of heterogeneous systems

  • Future work
  • Fix bottlenecks for increased scalability
  • Tree-style reduction
  • GT200-based GPUs -> zero-copy and simultaneous

communication and computation

  • Automatic data buffer sizing
slide-25
SLIDE 25

25

  • Dep. of Biomedical Informatics

HPC Lab bmi.osu.edu/hpc IEEE Cluster – PPAC - August 31, 2009 Timothy Hartley “GPU Clusters for SAR”

Thanks

  • Research at the HPC lab is funded by
  • Questions?