Generalized Significance in Scale Space: The GS3 Package Daniel V. - - PowerPoint PPT Presentation

Background Methods & Examples HSRL Data Conclusions & Future Work

Generalized Significance in Scale Space: The GS3 Package

Daniel V. Samarov

Statistical Engineering Division Information Technology Laboratory National Institute of Standards and Technology

July 21, 2010

1

Background Methods & Examples HSRL Data Conclusions & Future Work

Table of Contents

1

Background

2

Methods & Examples Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

3

HSRL Data

4

Conclusions & Future Work

2

Background Methods & Examples HSRL Data Conclusions & Future Work

Green House Gas (GHG) Emission Measurement

NIST developing technology & standards for remote sensing of GHG’s

3

Background Methods & Examples HSRL Data Conclusions & Future Work

Green House Gas (GHG) Emission Measurement

NIST developing technology & standards for remote sensing of GHG’s DIAL for distributed sources DIfferential Absorbtion LIDAR Range resolved, column integrated measurements

3

Background Methods & Examples HSRL Data Conclusions & Future Work

Green House Gas (GHG) Emission Measurement

NIST developing technology & standards for remote sensing of GHG’s DIAL for distributed sources DIfferential Absorbtion LIDAR Range resolved, column integrated measurements

3

Background Methods & Examples HSRL Data Conclusions & Future Work

HSRL

DIAL technology not quite ready for primetime. Collaboration w/ NASA

4

Background Methods & Examples HSRL Data Conclusions & Future Work

HSRL

DIAL technology not quite ready for primetime. Collaboration w/ NASA HSRL High Spectral Resolution LIDAR Similar technology/data Validation of Calipso satellite measurements

4

Background Methods & Examples HSRL Data Conclusions & Future Work

HSRL

DIAL technology not quite ready for primetime. Collaboration w/ NASA HSRL High Spectral Resolution LIDAR Similar technology/data Validation of Calipso satellite measurements

Hair et al. (2008)

Data graciously provided by NASA Langley Research Center

4

Background Methods & Examples HSRL Data Conclusions & Future Work

Challenges associated w/ HSRL & DIAL data

5

Background Methods & Examples HSRL Data Conclusions & Future Work

Challenges associated w/ HSRL & DIAL data

Highly variable

5

Background Methods & Examples HSRL Data Conclusions & Future Work

Challenges associated w/ HSRL & DIAL data

Highly variable Subtle local structure

Hair et al. (2008) 5

Background Methods & Examples HSRL Data Conclusions & Future Work

Challenges associated w/ HSRL & DIAL data

Highly variable Subtle local structure

Hair et al. (2008)

Large (∼ 300 × 30, 000)

5

Background Methods & Examples HSRL Data Conclusions & Future Work

Challenges associated w/ HSRL & DIAL data

Highly variable Subtle local structure

Hair et al. (2008)

Large (∼ 300 × 30, 000) Goals Estimate concentration (derivative) Calculate uncertainty

5

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Local Polynomial Regression (LPR)

Natural choice for derivative estimation: LPR (Fan & Gijbels (1995))

6

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Local Polynomial Regression (LPR)

Natural choice for derivative estimation: LPR (Fan & Gijbels (1995)) Pros Provides derivative estimate Locally adaptive Many other appealing properties

6

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Local Polynomial Regression (LPR)

Natural choice for derivative estimation: LPR (Fan & Gijbels (1995)) Pros Provides derivative estimate Locally adaptive Many other appealing properties Cons Challenge in 2(>)-d: bandwidth choice (in particular local) Speed Exploratory analysis

6

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Local Polynomial Regression (LPR)

Natural choice for derivative estimation: LPR (Fan & Gijbels (1995)) Pros Provides derivative estimate Locally adaptive Many other appealing properties Cons Challenge in 2(>)-d: bandwidth choice (in particular local) Speed Exploratory analysis The GS3 package provides a solution

6

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Scale Space

Scale space (Chaudhuri & Marron (2000)) a good starting point. Consider the following example:

7

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Scale Space

Scale space (Chaudhuri & Marron (2000)) a good starting point. Consider the following example: Many instances where a fit desired

7

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Scale Space

Scale space (Chaudhuri & Marron (2000)) a good starting point. Consider the following example: Many instances where a fit desired

7

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Scale Space

Scale space (Chaudhuri & Marron (2000)) a good starting point. Consider the following example: Many instances where a fit desired However, good practice to look at multiple smooths

7

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Scale Space

Scale space (Chaudhuri & Marron (2000)) a good starting point. Consider the following example: Many instances where a fit desired However, good practice to look at multiple smooths Scale space studies a“family”of smooths

7

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

RODEO

RODEO (Wasserman & Lafferty (2008)) greedy algorithm for traversing “scale space surface” Algorithm

8

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

RODEO

RODEO (Wasserman & Lafferty (2008)) greedy algorithm for traversing “scale space surface” Algorithm At a point, stay?

8

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

RODEO

RODEO (Wasserman & Lafferty (2008)) greedy algorithm for traversing “scale space surface” Algorithm At a point, stay?

• r move?

8

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

RODEO

RODEO (Wasserman & Lafferty (2008)) greedy algorithm for traversing “scale space surface” Algorithm At a point, stay?

• r move?

i.e. change from ˆ mh(x) to ˆ mh(x) significant?

8

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

RODEO

RODEO (Wasserman & Lafferty (2008)) greedy algorithm for traversing “scale space surface” Algorithm At a point, stay?

• r move?

i.e. change from ˆ mh(x) to ˆ mh(x) significant? Z = ∂ ˆ

mh(x) ∂h

, test |Z| >

• 2 log(n)Var(Z)

8

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

RODEO

RODEO (Wasserman & Lafferty (2008)) greedy algorithm for traversing “scale space surface” Algorithm At a point, stay?

• r move?

i.e. change from ˆ mh(x) to ˆ mh(x) significant? Z = ∂ ˆ

mh(x) ∂h

, test |Z| >

• 2 log(n)Var(Z)

NB: Var(Z) ∼ σ2

σ2 unknown population parameter

8

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Illustration of RODEO in Scale Space

For σ = 0.025

9

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Results

Produces sensible smooth & bandwidths

RODEO Solution Path, σ = 0.025

log2(bandwidth) x −0.5 −2.5 −4.5 −6.5 −8.4 0.25 0.97 1.7 2.42 3.14 −0.5 0.0 0.5 1.0

|Z|/(σ 2log(n)||g||2)

log2(bandwidth) x −0.5 −2.5 −4.5 −6.5 −8.4 0.25 0.97 1.7 2.42 3.14 2 4 6 8 10 12 0.5 1.0 1.5 2.0 2.5 3.0 −1.0 −0.5 0.0 0.5 1.0

Final fit

x y Actual Predicted 0.5 1.0 1.5 2.0 2.5 3.0 0.00 0.10 0.20 0.30

Final bandwidths

x bandwidth

10

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Back to σ

Looking for a fit, could estimate σ

11

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Back to σ

Looking for a fit, could estimate σ However, can also intrepet σ in similar way to h

Generalized (adaptive) scale space

log2(σ) x −1.5 −3.2 −5 −6.7 −8.4 0.25 0.97 1.7 2.42 3.14 −1.0 −0.5 0.0 0.5 1.0

Scale space

log2(bandwidth) x −0.5 −2.5 −4.5 −6.5 −8.4 0.25 0.97 1.7 2.42 3.14 −1.0 −0.5 0.0 0.5 1.0

11

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Higher dimensions

d > 1 Straightforward, calculate Zj, j = 1, . . . , d & Var(Zj) (∼ σ) Same greedy approach used along each dimension

Generalized Scale Space Scale Space 12

Background Methods & Examples HSRL Data Conclusions & Future Work Local Polynomial Regression Scale Space & RODEO Generalized Scale Space & d > 1 Algorithm Speed

Computation

Speed of Bandwidth Selection Use linear binning ideas (Fan & Marron (1994), Wand (1995)) Mi, i = 1, . . . , d grid size & T = # candidate bandwidths Selection takes O(dTM1 log M1 . . . Md log Md) Compared to smooth w/ global bandwidth, O(M1 log M1 . . . Md log Md)

13

Background Methods & Examples HSRL Data Conclusions & Future Work

Illustration of Generalized Scale Space on HSRL data

14

Background Methods & Examples HSRL Data Conclusions & Future Work

Conclusions & Future Work

Conclusions Provided R functions for 1, 2 & 3D binned LLR Implementation/development of method for fast local bandwidth estimation Generalization of scale space

15

Background Methods & Examples HSRL Data Conclusions & Future Work

Conclusions & Future Work

Conclusions Provided R functions for 1, 2 & 3D binned LLR Implementation/development of method for fast local bandwidth estimation Generalization of scale space Future Work (currently in progress) Local variance function estimation Spatial dependence

15