Towards a fully automated, high-performance pipeline for stereo - - PowerPoint PPT Presentation

▶
towards a fully automated high performance pipeline for
SMART_READER_LITE
LIVE PREVIEW

Towards a fully automated, high-performance pipeline for stereo - - PowerPoint PPT Presentation

Aug 22, 2023 647 likes •881 views

Towards a fully automated, high-performance pipeline for stereo reconstruction from earth observing satellites Dave Kelbe, PhD 1 Jeff Nichols, PhD 2 David Page, PhD 1 Devin White, PhD 1 Andrew Hardin 3 Lawrence MacIntyre 2 1 Geographic Data

slide-1
SLIDE 1

ORNL is managed by UT-Battelle for the US Department of Energy

Towards a fully automated, high-performance pipeline for stereo reconstruction from earth observing satellites

Dave Kelbe, PhD1 Jeff Nichols, PhD2 David Page, PhD1 Devin White, PhD1 Andrew Hardin3 Lawrence MacIntyre2

1Geographic Data Sciences Team

Geographic Information Science and Technology Group

2Cyber and Information Security Group 3Scalable and High Performance Geocomputation Team

Geographic Information Science and Technology Group

Oak Ridge National Laboratory

slide-2
SLIDE 2

GTC 2017 Kelbe

slide-3
SLIDE 3

GTC 2017 Kelbe

Scientific Foundation: Random dot stereograms

Normal Viewing Conditions Focus Convergence

slide-4
SLIDE 4

GTC 2017 Kelbe

Scientific Foundation: Random dot stereograms

Cross-Eyed Viewing Conditions Convergence Focus

slide-5
SLIDE 5

GTC 2017 Kelbe

Stereo reconstruction from earth observing satellites

  • Scientific Foundation: 3D illusions
  • Challenge and necessity of obtaining 3D surface data
  • Historical and technological motivation
  • Scientific foundation

– Development Concepts – Hardware – Pipeline

  • Technological solution: kernels
  • Current system performance
slide-6
SLIDE 6

GTC 2017 Kelbe

High Resolution Global Coverage Low Resolution Limited coverage

Shuttle Radar Topography Mission (SRTM) Airborne laser scanning UAV Structure from Motion ORNL Fortuitous Stereo Pipeline SRTM WorldDEM

Challenge: High-resolution, global surface data are scarce

ASTER GDEM (Stereo)

slide-7
SLIDE 7

GTC 2017 Kelbe

High-resolution global surface data are foundational

  • Photogrammetry is a

3-dimensional problem

X Z l s

Low-resolution surface High-resolution surface Terrain ortho True ortho

Available terrain estimate

slide-8
SLIDE 8

GTC 2017 Kelbe

Isn’t this a solved problem?

Wild A7 Stereoplotter (1952-1972) ”Easy Button” Software (2010’s - ) GPU Computing

slide-9
SLIDE 9

GTC 2017 Kelbe

Isn’t this a solved problem?

slide-10
SLIDE 10

GTC 2017 Kelbe

Scientific Foundation: Core development concepts

  • Use open source, in-house, or government-furnished s/w

libraries,

  • Leverage well-established photogrammetric and computer vision

techniques to reduce risk,

  • Expose components as services,
  • Use high performance computing (HPC) architectures,
  • Strictly adhere to and take full advantage of metadata standards

to enable interoperability, and

  • Keep the system as flexible as possible through extensive use
  • f modular frameworks.
slide-11
SLIDE 11

GTC 2017 Kelbe

Technological Solution: System Hardware

CPU/GPU hybrid architecture

  • Dell C4130 HPC Node
  • Each node has:
  • Dual Intel Xeon

processors:

  • 24 physical cores
  • 48 hyperthreaded
  • 256GB of RAM
  • 4 Tesla K80s
  • Used as 8 GPUs with

12GB each

slide-12
SLIDE 12

GTC 2017 Kelbe

Scientific Foundation: Pipeline

?

Left image Right image Matching cost computation Cost Aggregation Disparity computation Disparity refinement Stereo intersection Point Cloud Epipolar Rectification Registration GPU-accelerated Semiglobal Matching

slide-13
SLIDE 13

GTC 2017 Kelbe

i j i j pixel

CPU: Matching Cost Computation: Naïve

i j

abs(

  • )

=

for(pixel){ for(disparity){ } } costs costs(i,j)[d] = abs(left(i,j) – right(i-d,j)) left right i-d

slide-14
SLIDE 14

GTC 2017 Kelbe

pixel

GPU: Matching Cost Computation: Naïve

// Divide image into 2D grid pattern THREADS_PER_BLOCK.x = sqrt(1024); THREADS_PER_BLOCK.y = sqrt(1024); left // Index into i and j i = blockIdx.x * blockDim.x + threadIdx.x j = blockIdx.y * blockDim.y + threadIdx.y blockIdx.x 0 1 2 3 blockIdx.y 0 1 2 i j N_BLOCKS.x = sizex + THREADS_PER_BLOCK.X −1 THREADS_PER_BLOCK.X ; N_BLOCKS.y = sizey + THREADS_PER_BLOCK.Y −1 THREADS_PER_BLOCK.! threadIdx.x

0 1 2 3 … 31

slide-15
SLIDE 15

GTC 2017 Kelbe

i j i j pixel

CPU: Matching Cost Computation: Census with Hamming

i j

for(pixel){ for(disparity){ } } }

costs

for(windowSize){

left right i-d

xor( , ) = sum( )

slide-16
SLIDE 16

GTC 2017 Kelbe

CPU: Cost Aggregation

i j costs i j accumulated

for(pixel){ for(disparity){ for(disparity){ }}}

current prior local + min( )

slide-17
SLIDE 17

GTC 2017 Kelbe

GPU: Cost Aggregation

i j costs i j accumulated

for(pixel){ for(disparity){ for(disparity){ }}}

… …

// Divide image into 2D grid pattern THREADS_PER_BLOCK.x = 1024/dispartyRange; THREADS_PER_BLOCK.y = disparityRange; current prior local disparityRange

slide-18
SLIDE 18

GTC 2017 Kelbe

Performance Comparison

48 CPU 8 GPU

2 4 6 8 10 12 14 16 Computation Time (Seconds) Pipeline Timings Startup Cost Calculation Cost Aggregation Disparity Computation GPU Overhead * GPU accelerated

slide-19
SLIDE 19

GTC 2017 Kelbe

Our solution: Fortuitous stereo 3D reconstruction

slide-20
SLIDE 20

GTC 2017 Kelbe

slide-21
SLIDE 21

GTC 2017 Kelbe

Conclusions

  • Motivation: Global, high resolution surface

data are critical for photogrammetric preprocessing

  • Scientific Foundation: Computationally

intensive, but class-leading Semi-Global Matching algorithm (SGM)

  • Technological Solution:

– GPU acceleration makes processing of rapid streams of satellite imagery viable

slide-22
SLIDE 22

GTC 2017 Kelbe

Thank you

Dave Kelbe, PhD Oak Ridge National Laboratory kelbedj@ornl.gov