Spectro-Perfectionism: An Algorithmic Framework for Photon - - PowerPoint PPT Presentation

▶
spectro perfectionism an algorithmic framework for photon
SMART_READER_LITE
LIVE PREVIEW

Spectro-Perfectionism: An Algorithmic Framework for Photon - - PowerPoint PPT Presentation

Jan 08, 2023 369 likes •935 views

Spectro-Perfectionism: An Algorithmic Framework for Photon Noise-Limited Extraction of Optical Fiber Spectroscopy Adam S. Bolton The University of Utah Department of Physics & Astronomy Exoplanet PRVs - PSU - 2010 Aug 19 Beware of... o

slide-1
SLIDE 1

Spectro-Perfectionism: An Algorithmic Framework for Photon Noise-Limited Extraction of Optical Fiber Spectroscopy

Adam S. Bolton The University of Utah Department of Physics & Astronomy Exoplanet PRVs - PSU - 2010 Aug 19

slide-2
SLIDE 2

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Beware of...

  • What you think you know about LSFs

and cross-sectional profiles

  • Extragalactic astronomers proffering advice
  • Fake data
slide-3
SLIDE 3

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Spectro-Perfectionism:

What is the right way to go from this:

slide-4
SLIDE 4

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Spectro-Perfectionism:

What is the right way to go from this: ... to this: ?

slide-5
SLIDE 5

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Spectro-Perfectionism:

Bolton & Schlegel (2010, PASP , 122, 248)

slide-6
SLIDE 6

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Hasn’t this problem been solved?

Yes, sort of...

slide-7
SLIDE 7

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Hasn’t this problem been solved?

“Perfectionism is a disease”

  • PLS
slide-8
SLIDE 8

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Why do I care?

www.SLACS.org

slide-9
SLIDE 9

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Why might you care?

You’re already forward-modeling your spectra. Why not forward-model your raw data, too? Signal-to-noise regimes: SNR ~ 100: systematics limited SNR ~ 10: statistics limited SNR ~ 1: systematics limited Astronomy as experimental physics: we don’t control the accelerator, so best to control and understand the detector well! Spectra get fainter; sky stays as bright as ever.

slide-10
SLIDE 10

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Systematics of sky subtraction

slide-11
SLIDE 11

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Systematics of sky subtraction

slide-12
SLIDE 12

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

What do we want in an extraction?

  • Define in terms of objective scalar optimization
  • Generate noise-limited model of all 2D frames
  • Allow optimal weighting
  • Do not degrade resolution
  • Characterize resolution accurately
  • Avoid correlations in extracted 1D samples
  • Propagate errors correctly (for correct chi^2)
  • Preserve these properties in multi-frame coadds
  • Allow foreground estimation and subtraction in the

presence of optical non-uniformities

  • Deliver something that fits an astronomer’s

understanding of “the extracted spectrum”

  • Make it easy to implement
slide-13
SLIDE 13

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Boxcar extraction

slide-14
SLIDE 14

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Boxcar extraction

  • Draw two lines
  • Sum enclosed counts
  • Call that your spectrum

The “quick and dirty” method.

slide-15
SLIDE 15

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Boxcar scorecard

Define in terms of objective scalar optimization Generate noise-limited model of all 2D frames Allow optimal weighting Do not degrade resolution Characterize resolution accurately Avoid correlations in extracted 1D samples Propagate errors correctly (for correct chi^2) Preserve these properties in multi-frame coadds Allow foreground estimation and subtraction in the presence of optical non-uniformities Deliver something that fits an astronomer’s understanding of “the extracted spectrum” Make it easy to implement

slide-16
SLIDE 16

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Optimal extraction

slide-17
SLIDE 17

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

  • Determine cross-sec’n
  • Weighted amplitude fit
  • Call that your spectrum

The current standard in extraction (e.g., SDSS: Burles & Schlegel)

Optimal extraction

Hewett et al. 1985; Horne 1986

slide-18
SLIDE 18

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Optimal-extraction scorecard

Define in terms of objective scalar optimization Generate noise-limited model of all 2D frames Allow optimal weighting Do not degrade resolution Characterize resolution accurately Avoid correlations in extracted 1D samples Propagate errors correctly (for correct chi^2) Preserve these properties in multi-frame coadds Allow foreground estimation and subtraction in the presence of optical non-uniformities Deliver something that fits an astronomer’s understanding of “the extracted spectrum” Make it easy to implement (almost)

slide-19
SLIDE 19

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

The general problem Ajk ( fk + sk ) = pj + nj + bj Ajk: Calibration matrix fk: Input flux vector sk: Input background vector pj: Pixel count (data) vector nj: Pixel noise vector bj: Internal background vector

slide-20
SLIDE 20

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

The general problem Ajk ( fk + sk ) = pj + nj + bj Ajk: Calibration matrix

  • Wavelength solution
  • Spectral trace solution
  • Cross-sectional profile
  • Relative pixel response
  • Line-spread function
  • Relative fiber response
  • Flux calibration
  • Camera aberrations

A sparse matrix that unifies and extends:

slide-21
SLIDE 21

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Extraction as image modeling

“data” log10 [ pixval / <pixval>]

Model fiber PSF for SDSS1 @ 8500Å

slide-22
SLIDE 22

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

How do you extract an emission line?

Classic optimal extraction can only be correct when the spectrograph PSF is a separable function of x and y.

slide-23
SLIDE 23

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Extraction as image modeling

row model “data” log10 [ pixval / <pixval>]

slide-24
SLIDE 24

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Extraction as image modeling

2D model row model “data” log10 [ pixval / <pixval>]

slide-25
SLIDE 25

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

2D extraction model residuals

2D model row model pixval / <pixval>

slide-26
SLIDE 26

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Deconvolution and reconvolution

slide-27
SLIDE 27

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

The general problem Ajk ( fk + sk ) = pj + nj + bj Ajk: Calibration matrix fk: Input flux vector sk: Input background vector pj: Pixel count (data) vector nj: Pixel noise vector bj: Internal background vector

slide-28
SLIDE 28

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Put resolution back into spectrum

The formal (deconvolved) solution: f = ( AT N-1 A )-1 AT N-1 p Inverse covariance matrix of deconvolved spectrum: C-1 = AT N-1 A Take unique non-negative square root of this matrix: C-1 = QQ Normalize along rows & factor out a diagonal matrix: C-1 = RT C-1 R By consequence: C = R C RT The reconvolved spectrum: what would have been

  • bserved by a 1D spectrograph with same resolution:

f = R f

slide-29
SLIDE 29

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Put resolution back into spectrum

The formal (deconvolved) solution: f = ( AT N-1 A )-1 AT N-1 p Inverse covariance matrix of deconvolved spectrum: C-1 = AT N-1 A Take unique non-negative square root of this matrix: C-1 = QQ Normalize along rows & factor out a diagonal matrix: C-1 = RT C-1 R By consequence: C = R C RT The reconvolved spectrum: what would have been

  • bserved by a 1D spectrograph with same resolution:

f = R f Note analogy Uncorrelated errors Band diagonal

slide-30
SLIDE 30

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Deconvolution and reconvolution

slide-31
SLIDE 31

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Comparative resolution

  • w. r. t. boxcar
slide-32
SLIDE 32

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

To make things interesting, add:

  • Noise,
  • Variable fiber PSF

,

  • Multiple frames with flexure/dither,
  • “Sky”,
  • Fiber-to-fiber crosstalk
slide-33
SLIDE 33

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

To make things interesting, add:

  • Noise,
  • Variable fiber PSF

,

  • Multiple frames with flexure/dither,
  • “Sky”,
  • Fiber-to-fiber crosstalk

Can do extraction, coaddition, and sky subtraction in one shot!

slide-34
SLIDE 34

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Multi-frame, multi-fiber simulated data

slide-35
SLIDE 35

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Multi-frame, multi-fiber simulated data

Sky #1 Sky #2 Sky #3 Object #1 Object #2

slide-36
SLIDE 36

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Multi-frame, multi-fiber simulated data

Objflux = Skyflux / 20 ObjSNR ≈ 5 (per extracted sample, sky-noise limited)

slide-37
SLIDE 37

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Sky model decomposed & removed

(Grayscale stretch X 40 relative to previous)

Sky spectrum is modeled “upstream” from

  • ptical heterogeneities between fibers
slide-38
SLIDE 38

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

All models removed

Consistent with pure noise

slide-39
SLIDE 39

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Extracted objects + skies

Sky scaled down by a factor of 20 in plot RMS error- scaled residuals of unity

slide-40
SLIDE 40

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Spectro-perfectionism scorecard

Define in terms of objective scalar optimization Generate noise-limited model of all 2D frames Allow optimal weighting Do not degrade resolution Characterize resolution accurately Avoid correlations in extracted 1D samples Propagate errors correctly (for correct chi^2) Preserve these properties in multi-frame coadds Allow foreground estimation and subtraction in the presence of optical non-uniformities Deliver something that fits an astronomer’s understanding of “the extracted spectrum” Make it easy to implement

slide-41
SLIDE 41

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

The biggest headache (for me, at least):

Fiber-to-fiber cross-talk couples all spectra. For each BOSS spectrograph-plate, we have: 500 spectra × 6000 sampling points × 4 frames ⇒ 12 Million coupled equations to solve! Fortunately the matrix is sparse... (Sampling also swept under rug here, but see paper...)

slide-42
SLIDE 42

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

The challenge to calibration and design

Current calibration facilities may not permit a sufficiently accurate determination of Ajk => New calibration regimes and equipment (high- wattage monochrometer or tunable laser) Ultimately calls for a full integration of data analysis software with instrumental design software => Optimize scientific metrics in hardware => Tune instrument directly from data => “Use what you know” during analysis

(c.f. Stubbs & Tonry 2006)

slide-43
SLIDE 43

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

The sociological challenge

Site selection: Multi-year testing, remote locations, etc. Telescope: As large, reflective, and well-focused as possible Instrument: Expensive design, coatings, high-QE CCDs Data calibration and extraction: Somebody will do something at some point... What’s wrong with this picture?

slide-44
SLIDE 44

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Summary

  • Current extraction algorithms are inaccurate at a level

that significantly degrades faint-object fiber spectra

  • This problem can be solved with correct 2D modeling
  • Resolution is a preservable native attribute of raw data
  • Extracted covariance can be made diagonal
  • Extraction, coaddition, and sky subtraction in one shot
  • chi^2 against spectra <=> chi^2 against raw data
  • Immediate application for SDSS-III BOSS
  • Very accurate calibration: difficult but important
  • Computational challenge is significant
  • Check it out: Bolton & Schlegel 2010, PASP

, 122, 248

slide-45
SLIDE 45

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

Thank You!

slide-46
SLIDE 46

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-47
SLIDE 47

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-48
SLIDE 48

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-49
SLIDE 49

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-50
SLIDE 50

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-51
SLIDE 51

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-52
SLIDE 52

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-53
SLIDE 53

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-54
SLIDE 54

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-55
SLIDE 55

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines

slide-56
SLIDE 56

Adam S. Bolton Exoplanet PRVs - PSU - 2010-08-19

B-splines