8. Mid-level processing (data preparation) Andr Jalobeanu LSIIT / - - PowerPoint PPT Presentation

Introduction to Astronomical Image Processing

• 8. Mid-level processing

(data preparation)

André Jalobeanu LSIIT / MIV / PASEO group

• Jan. 2006

lsiit-miv.u-strasbg.fr/paseo Master ISTI / PARI / IV

PASEO

Mid-level processing: data preparation

Data presentation

Enhanced visualization Color display of multidimensional data Multiresolution vision model

Dimensionality reduction

Dimensionality reduction methods, Band fusion

Object extraction and signal/noise separation

SNR maximization via binning Significant information detection

Combining multiple observations: data fusion

Frame co-addition, mosaicing Super-resolution, multiframe restoration

Data presentation

Learn how to better display grayscale or RGB color images Get acquainted to problems related to multispectral image visualization See how nonlinear transforms and multiscale representations can help visualize lots of information at once

Enhanced visualization

single band (grayscale) images

High dynamic range, difficult to visualize

• linear pixelwise transforms (e.g. histogram clipping)
• nonlinear pixelwise transforms (e.g. log-scale)
• multiple pixel operations (e.g. opacity filtering, unsharp mask)
• grayscale to RGB mapping: false color display

histogram clipping M51 in false color (C. Buil)

Enhanced visualization

single band (grayscale) images

Opacity masking (M51) Unsharp masking (Sun spot)

X+k.(X-X*g) X.f(X*g)

amplify details (image - blurred image) reduce high pixel intensities,

• pacity mask=

blurred image

• C. Buil
• C. Buil

g = Gaussian blur

Enhanced visualization

3-band (RGB) images

๏ Color space manipulations (e.g. HSV space)

• Saturation adjustment (S)

Compensate for overlapping filters (poor color image)

• Color cycling (H)

Try different colors for a better visual segmentation

• Nonlinear intensity transform with color preservation (V)

Useful for high dynamic range images

Before and after saturation adjustment

• C. Buil

Saturation-enhanced “true colors” (400, 560, 910nm)

HSV space

• C. Buil

Color display of multidimensional data

๏ Multiband image reduction & display [Petremand 04]

• PCA or ICA reduction (keep at least 6 bands)
• Markovian segmentation: label map
• Discriminant Factor Analysis: color (H,S)

Work in the HSV (Hue Saturation Value) space

• PCA for each class: V
• HSV to RGB conversion

6 reduced bands (from 48) Label map and color display

High dimensionality of multi/hyperspectral images: Impossible to visualize them as they are!

Reduction to 3 bands (RGB) using PCA, ICA... insufficient (stationary model)

Multiscale Vision Model

๏ Use a multiresolution transform (wavelets)

• Select significant coefficients (above noise)
• Grow regions defined by the significant coefficient location
• Keep regions according to the inter-scale, region-based connectivity

(image segmentation)

[Bijaoui 95]

Galaxy MVM segmentation Connectivity

Dimensionality reduction

Grasp the ideas behind classical dimensionality reduction methods Understand why such methods are not well-suited to astronomy Get to know recent techniques used to merge multispectral data into a single band image

Dimensionality reduction methods

๏ Principal Component Analysis (PCA) methods

Maximum variance subspace, Gaussian assumptions

• Simple PCA - Gaussian cluster, no noise, one step
• Probabilistic PCA, Factor Analysis - add some noise (indep.)
• Mixture of PCA - mixture of PCA models: multiple clusters, nonlinearity

๏ Independent Component Analysis (ICA)

Non-Gaussian components assumption

• Simple ICA - maximize the nongaussianity
• Probabilistic ICA - add some noise
• Projection pursuit - more general: maximize projection index
• Mixture of ICA - multiple clusters, nonlinearity

linear embedding subspace

Linear vs. nonlinear embedding subspace

iterative algorithms, EM used in probabilistic approaches

Major drawbacks:

• ◆ Stationarity assumption
• ◆ No image formation model (e.g. blur)

linear embedding subspace

๏ Problems specific to astronomical images

• Spatially adaptive behavior
• Dark background & photon noise
• Sources multiplied by spatially variable density (e.g. nebulae)
• Compound sources (e.g. star clusters, galaxies)
• Complex sources (e.g. stars)

๏ Problems related to Integral Field Spectroscopy

• Very large number of bands (thousands)

One spectrum for each spaxel, contains lots of information that needs to be preserved!

• Very small number of spatial samples (until Muse, 2012)

Dimensionality reduction of deep field images

Synthetic image 2-band histogram Color histogram (Hue,Saturation)

Better goal: Source separation or decorrelation

• Find minimum number of independent sources
• To preserve information, may be larger than the number of bands!
• Determine their spatial distribution (spatial consistency)

Band fusion

๏ Merge multidimensional data into a single band

SNR maximization & detection purposes

• Simple averaging (naive, inaccurate)
• Weighted averaging

Optimal averaging, stationary noise

• Spatially adaptive averaging
• Use image space information

(e.g. saturation, bad pixels)

• Use a multiscale approach based on

wavelet transforms

Wavelet domain band fusion using Hidden Markov Trees (combined with Van Cittert deconvolution) [Flitti 05]

Object extraction and signal/noise separation

Grasp the principles of SNR enhancement using spatial grouping Become familiar with the significant information extraction problem and the related challenges

SNR maximization via spatial binning

๏ Uniform binning

• Sum or average pixel values within

predefined super-pixels

๏ Spatially adaptive binning

• Find image space subdivision such that

SNR=constant (prescribed SNR)

• Quadtree plane partition
• Voronoi tessellation

X-ray data binning, by S. Diehl & T. Statler Voronoi tessellation

Maximize the SNR by averaging samples

(Central Limit Theorem: σ →σ/√n)

Quadtree partition

Significant information detection

๏ Nonparametric approaches

• Strong denoising (ensure a minimum false alarm rate)

Use existing methods and minimize the residual noise

• Enhanced visualization of significant transform coefficients

(e.g. Multiscale Vision Model)

๏ Parametric methods

• Supervised object detection
• Use template matching to detect predefined patterns
• Recursive object detection/subtraction (e.g. CLEAN)
• Feature extraction (e.g. local maxima)
• Statistical fitting (parametric objects), decision theory
• Unsupervised object extraction & optimal representation
• Pixon approach [Pina-Puetter 93]:

Set of parametric shapes (dictionary), simplest representation → Keep the parameters (shape, scale, location)

• Fully Bayesian approaches, to be developed
• Both spatial and spectral methods (for IFS), to be developed

Combining multiple

• bservations: data fusion

Understand how to combine multiple images into a single one See how super-resolution can help preserve information and extrapolate a limited bandwidth Get acquainted to some techniques used to increase SNR, FOV and bandwidth in astronomy

Data fusion: introduction

๏ Multisource data fusion objectives

• Optimally combine all observations into a single image

Co-add or build a mosaic, depending on the overlap

• Preserve all the information from the original data set

Increase resolution if needed, compute the uncertainties

• Optional: enhance the image quality

Denoise or deblur depending on the degradation

๏ Possible approaches

• Forward methods

Shift-and-add, register-and-add, simple resampling schemes, drizzle

• Inverse methods

Bayesian super-resolution, probabilistic fusion, multiframe deconvolution

Problem: lots of data, same object!

Usually, images are recorded with various:

• ◆ pose parameters (position, orientation)
• ◆ sensors (resolution, noise, bad pixels)
• ◆ observing conditions (transparency, seeing)
• ◆ telescopes (PSF, distortions)

Frame co-addition in deep-field imaging

increase SNR & dynamic range

๏ Register-and-add

• Apply geometric corrections: resample input images

Avoid simple interpolation schemes!

• Add resulting images

Use weighted sum to account for noise variations Apply rank filters to remove cosmic rays & bad pixels

๏ Drizzle

• Designed to handle undersampled HST images [Fruchter 96]
• Shrink input pixels, project onto finer output grid (scale ≈0.5-0.7)

Perform geometric corrections, including distortions

• Add: Image update iteration (add a drop)

X ← (awY+WX)/(aw+W), W ← aw+W

Y = drop value (pixel value in the input image) a = intersection btw. drop and fine output grid w = user-defined pixel weight (significance)

potential problems: noise correlation, image blurring, sampling...

Prerequisite: accurate camera calibration (estimate the registration parameters)

X Y

Image mosaicing

increase the FOV

๏ Same as image co-addition, with some specificities

• In all cases:
• Accurate camera calibration using overlapping areas
• Image normalization (flat, transparency, exposure) to avoid boundary effects
• Register-and-add

Apply geometric corrections (registration), add images in overlapping areas

• Drizzle

Shrink pixels, project (geometric transform), add drops to finer pixels

O’Dell / NASA

Image mosaicing in planetary and wide field astronomical imaging

Output pixel size < input PSF FWHM / 2

Super-resolution

de-aliasing and bandwidth extrapolation

๏ Go beyond the sampling resolution - multiple images

Stack of generally undersampled images

• Image reconstruction & deconvolution techniques
• 3 steps: deconvolution, registration, averaging
• Combined: deconvolution with specificities (multiplicity, sampling):

multiple data terms, explicit subsampling, batch or recursive processing

• Super-resolution without deblurring (register-and-add, drizzling)

๏ Go beyond the diffraction limit - single image

One blurred, oversampled image (could be subsampled/binned without loss)

• Model-based approaches

Use powerful image models based on prior knowledge! Reconstruction possible beyond cutoff frequency if near-black object [Donoho 92]

Super-resolution for point sources using pixons (1 image)

Pixon LLC

4x super-resolution from 16 images

[Willett 04]

Multiframe restoration

partial turbulence compensation

๏ Reconstruction & Deconvolution methods

Negligible phase perturbations: constant PSF, small shifts

• Specificities: multiple data terms, batch or recursive processing,

shift or undistort (space-variant shifts) before/after deblurring

๏ Handling phase perturbations

• Blind deconvolution methods (time-varying PSF)

Specificity: constant underlying image

• Speckle imaging
• power spectrum averaging (lost phase information)
• constrained power spectrum and phase reconstruction
• Phase diversity

Record in-focus and out-of-focus images simultaneously

Imaging through turbulence: random phase perturbations

Example of image sequence (star)

Incremental Poisson MAP restoration. One of the observed images, results with 10 and 50 images [Sheppard 98]

Further reading ๏ Enhanced visualization from the Iris tutorial

http://www.astrosurf.org/buil/iris/tutorial1/doc2_us.htm http://www.astrosurf.org/buil/iris/tutorial5/doc15_us.htm

๏ MultiColorViz (hyperspectral image display)

http://lsiit-miv.u-strasbg.fr/paseo/research.php#proj5

๏ Tutorials on dimensionality reduction

http://www.oid.ucla.edu/Webcast/ipam/

๏ Voronoi tessellation & adaptive binning

http://www.phy.ohiou.edu/~diehl/WVT/

๏ Dithering and Drizzling (A. Fruchter)

http://www-int.stsci.edu/~fruchter/dither/dither.html

๏ Phase reconstruction and phase diversity

http://www.phy.hw.ac.uk/~phyhic/Theory%20Pages/

๏ Super-Resolution (S. Borman)

http://www.seanborman.com/publications/