Production Methods for the F162009 Stable Lights Product Kimberly - - PowerPoint PPT Presentation

Production Methods for the F162009 Stable Lights Product

Kimberly Baugh, CIRES University of Colorado Boulder, Colorado, USA Email: kim.baugh@noaa.gov 1‐303‐497‐4452 Chris Elvidge, NOAA National Geophysical Data Center (NGDC) Boulder, Colorado, USA Email: chris.elvidge@noaa.gov 1‐303‐497‐4452

F162009 Global Stable Lights Product

• NGDC’s “Stable Lights” products have values which depict the

relative intensity of lit areas.

• Lighting deemed ephemeral has been removed and non‐lit areas

(background) have been set to zero.

DMSP‐OLS Data

DMSP‐OLS sensor has a unique capability to collect low‐light imagery.

• Polar orbiting
• 3000 km swath
• 2.7 km ground sample

distance

• Daily global coverage
• Two spectral bands:
• VIS (0.4‐1.1 um)
• TIR (10.5‐12.6 um)
• Visible band data values

range from 0‐63 (6‐bit data)

VIS TIR

Nighttime portion of orbit F16200901281215

Generation of v4b F162009 Stable Lights Product

1) OLS data for satellite F16, year 2009, is gridded to 30‐arc second grid. The highest quality data is kept for inclusion into the product. Highest quality means: mid‐swath, cloud‐free, zero lunar illuminance, dark nightttime (solar elevation < ‐15). 2) An analyst looks at each suborbit to manually mark aurora and gain changes for exclusion. 3) 30‐arc second grids are composited, creating a suite of files including an average visible band image and histograms of input visible band data for each grid cell. 4) Outlier removal process is performed to remove ephemeral lighting events such as fires. 5) Background removal is done which uses no‐light areas to generate local background statistics. A “lights mask” is output from this process. 6) Create stable lights image by applying the lights mask to original average visible band data. 7) Align stable lights image to Landscan population grid.

Processing DMSP‐OLS Orbits: Flag Bands

For each nighttime suborbit, a companion “flag” band is generated with bit‐ codes designating:

• daytime (solar elevation > ‐6)
• nighttime marginal

(‐15 < solarelevation < ‐6)

• zero lunar illuminance

(< 0.0005 lux)

VIS

Nighttime portion of orbit F16200901281215

FLAG

This entire suborbit was flagged as having zero lunar illuminance. Red: daytime Green: nighttime marginal Black: This area is considered high quality nighttime data and will be processed further.

Solar elevation angles are computed based on lat, lon, and time of each OLS pixel. Lunar illuminance is a function of lunar phase, azimuth, and elevation, which are also based

• n lat, lon, and time of each OLS

pixel.

Red: daytime Green: nighttime marginal Yellow: discarded by linescreening process Blue: edge‐of‐scan data Black: This area is considered high quality nighttime data and will be processed further.

Processing DMSP‐OLS Orbits: Linescreening

VIS

Nighttime portion of orbit F16200901281215

FLAG

• Suborbits containing

high quality nighttime data are screened by an analyst for aurora and abrupt gain changes.

• Analyst chooses a start

and end line of data to include for compositing.

• Data at edges of swath

are discarded due to increased noise and poorer geolocation (scan angle > 40.91) .

Processing DMSP‐OLS Orbits: Reprojection

VIS TIR

Nighttime portion of orbit F16200901281215

FLAG

OLS vis, tir and corresponding flag bands are gridded to 30 arc‐second grids, constrained to latitudes 65S‐75N. For clarity, only mid‐swath, linescreened data are shown.

Green: nighttime marginal

Processing DMSP‐OLS Orbits: Cloud Mask

• A cloud mask is generated by

comparing the reprojected OLS thermal band to a surface temperature grid.

• National Center for Environmental

Prediction (NCEP) creates global surface temperature grids at 0.5, 1.0, and 2.5 degree resolution at 6 hourly intervals.

• The 1.0 degree surface

temperature grids were used for the F162009 stable lights product.

00:00 UT NCEP grids for 2009/01/28 06:00 UT 12:00 UT 18:00 UT

Processing DMSP‐OLS Orbits: Cloud Mask

• Difference images are made as Diff = Surface Temp ‐ TIR.
• Due to the increased variability in land temperature

values, land and ocean regions are processed separately.

TIR NCEP Surface Temp. Diff (Land) Diff (Sea)

Processing DMSP‐OLS Orbits: Cloud Mask

• Thresholds are computed from the difference images in latitudinal tiles as

mean+N*stdev. Values greater than this threshold are flagged as clouds.

• At high latitudes the land/cloud temperatures converge, so the value of N

varies linearly with latitude from N=4 at +/‐15 to N=1 at +/‐60.

TIR Diff (Land) Diff (Sea) Cloud Mask

Stable Lights Processing: Compositing

• The 30‐arc second visible band

grids are masked to areas of cloud‐free, high‐quality nighttime data using the flag band.

• The masked grids are then

composited, creating a suite of files including an average visible band image, number of cloud‐ free observations used, and histograms of input visible band data for each grid cell.

Average Vis N Cloud‐free Observations Histograms are made for each grid cell

Stable Lights Processing: Outlier Removal

• Histograms for each output grid cell are analyzed.
• Observations are iteratively removed from the top end of the histogram until the

standard deviation of the observations has stabilized or >50% of the observations are removed (no convergence) Cell with fires. Process removed highest 10

• bservations.

Cell with town. Process removed highest

• bservation.

Stable Lights Processing: Outlier Removal

Average of Visible Band Observations Average After Outliers Removed

Stable Lights Processing: Outlier Removal

Avg Vis After Outlier Removal (F162009) Avg Vis Before Outlier Removal (F162009)

400 pixels 25 pixels

Tile 001 of 256 Tile 002 of 256 Tile 003 of 256

Stable Lights Processing: Background Removal

Background values vary significantly in the global outlier‐ removed average image, therefore local background threshold values are computed.

• 1. Areas known to be light‐free are chosen

by an analyst.

• 2. Using the outlier‐removed average visible

image: a) For each kernel of size 25X25, the 256 400X400 tiles containing this kernel are examined. b) Areas in the kernel with values greater than the maximum light‐free values from each tile are tallied as “greater than background”.

• 3. Stable lights mask is generated as areas

considered “greater than background” at least 40% of the time. Tile 016 of 256 Tile 119 of 256 Tile 256 of 256

Stable Lights Processing: Background Removal

Light‐free areas chosen by an analyst in red Resulting Stable Lights mask

Stable Lights Processing: Background Removal

Average After Outliers Removed Average w/ Background Removed

Stable Lights Processing: Align to Landscan Population Grid

A cross‐correlation technique is used to generate a “best‐fit” linear translation between a the Stable Lights grid and the Landscan Population grid.

Red‐>F162009 Stable Lights, Cyan‐>Landscan Population Grid

Florida: Before Alignment Florida: After Alignment Florida Keys: Before Alignment Florida Keys: After Alignment

F162009 Global Stable Lights Product

• NGDC’s “Stable Lights” products have values which depict the

relative intensity of lit areas.

• Lighting deemed ephemeral has been removed and non‐lit areas

(background) have been set to zero.

F162009 Global Stable Lights: Caveats

• The Stable Lights Outlier Removal process effectively removes most noise and

ephemeral lights such as fires and fishing boats.

• Lights from boats and fires which are present in over 50% of the cloud‐free
• bservations remain in the Stable Lights product.

Avg Vis Avg Vis after Outlier Removal Stable Lights Lights from boats Many lights from boats remain These boats are part of the Stable Lights product

F162009 Global Stable Lights: Caveats

• The Stable Lights Outlier Removal process will remove ANY ephemeral light.
• Some gas flares, towns with unstable power supplies, and seasonal

recreational areas (e.g. ski resorts) will not show up in the Stable Lights product.

Avg Vis Avg Vis after Outlier Removal Stable Lights Gas Flare Removed as ephemeral Absent from Stable Lights

• Coarse spatial

resolution of the OLS 2.7 km GSD 5+ km GIFOV

• OLS lights are larger

than sources on the ground

• “Overglow” surrounds

bright sources

• No visible band

calibration

• 6 bit quantization
• Urban centers saturate

in operational data

• No 3‐5 um band for fires

F162009 Global Stable Lights: Caveats

Contrast enhanced to show dim lighting

16:30 17:42 18:54 20:06 21:18 22:30

1/1/1992 27/9/1994 23/6/1997 19/3/2000 14/12/2002 9/9/2005 5/6/2008

Time

Date

DMSP Local Times at the Ascending Equatorial Crossing

F10 F11 F12 F13 F14 F15 F16 F17 F18

Version 4b Stable Lights Products

• Generated as calendar year products from

1992‐2009:

F10: 1992*, 1993, 1994 F12: 1994*, 1995, 1996, 1997, 1998, 1999, 2000 F14: 1997*, 1998, 1999, 2000, 2001, 2002 F15: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008* F16: 2004, 2005, 2006, 2007, 2008, 2009

*indicates partial year due to start/end of usable data

• Data available online at:

http://www.ngdc.noaa.gov/dmsp/downloadV4composites.html