Initial Validation of a Convective Weather Avoidance Model (CWAM) - - PowerPoint PPT Presentation

▶
initial validation of a convective weather avoidance
SMART_READER_LITE
LIVE PREVIEW

Initial Validation of a Convective Weather Avoidance Model (CWAM) - - PowerPoint PPT Presentation

Jan 23, 2023 414 likes •692 views

Initial Validation of a Convective Weather Avoidance Model (CWAM) in Departure Airspace Mikhail Rubnich and Rich DeLaura 30th Digital Avionics Systems Conference October, 18th 2011 MIT Lincoln Laboratory 999999-1 XYZ 10/22/2011 Contents

slide-1
SLIDE 1

999999-1 XYZ 10/22/2011

MIT Lincoln Laboratory

Initial Validation of a Convective Weather Avoidance Model (CWAM) in Departure Airspace

Mikhail Rubnich and Rich DeLaura 30th Digital Avionics Systems Conference October, 18th 2011

slide-2
SLIDE 2

MIT Lincoln Laboratory

999999-2 XYZ 10/22/2011

Contents

  • Goals and motivations
  • Automatic avoidance detection algorithm description
  • Analysis of results
  • Conclusions and future work
slide-3
SLIDE 3

MIT Lincoln Laboratory

999999-3 XYZ 10/22/2011

Contents

  • Goals and motivations
  • Automatic avoidance detection algorithm description
  • Analysis of results
  • Conclusions and future work
slide-4
SLIDE 4

MIT Lincoln Laboratory

999999-4 XYZ 10/22/2011

Motivations

  • The Route Availability Planning Tool (RAPT) - decision

support tool used to help controllers in route management has problems with over-warning and occasional under- warning when weather impacts are in terminal airspace

  • RAPT is using Convective Weather Avoidance Model

(CWAM) and an airspace use model

  • Therefore, CWAM in terminal airspaces needs to be

validated

slide-5
SLIDE 5

MIT Lincoln Laboratory

999999-5 XYZ 10/22/2011

Weather Avoidance Field* description

Convective Weather Avoidance Model Echo top (storm height) VIL (precipitation intensity) Departure Domain Weather Avoidance Field (WAF) (probability of pilot deviation )

* DeLaura, R., and Evans, J., “An Exploratory Study of Modeling Enroute Pilot Convective Storm Flight Deviation Behavior,”

Proceedings of the 12th Conference on Aviation, Range, and Aerospace Meteorology, Atlanta, 2006 Terminal boundary Enroute boundary Transition

slide-6
SLIDE 6

MIT Lincoln Laboratory

999999-6 XYZ 10/22/2011

Chicago and New York Airspaces

Chicago Airspace

30 minute cumulative traffic

Key: Departures Arrivals

New York Airspace

slide-7
SLIDE 7

MIT Lincoln Laboratory

999999-7 XYZ 10/22/2011

Methodology

  • Trajectories from Enhanced Traffic Management System

( ETMS ), WAF calculated using observed weather from Corridor Integrated Weather System (CIWS)

  • Calculated weather avoidance ratio using automatic

avoidance detection algorithm using 5 test days ( Chicago) and 8 test days ( New York ) from 2010

  • 489 weather avoidances and 523 weather intersections (

Chicago ), 1084 weather avoidances and 1337 weather intersections ( New York ) were identified and analyzed

  • WAF calibration using observed avoidance ratio
slide-8
SLIDE 8

MIT Lincoln Laboratory

999999-8 XYZ 10/22/2011

Contents

  • Goals and motivations
  • Automatic avoidance detection algorithm description
  • Analysis of results
  • Conclusions and future work
slide-9
SLIDE 9

MIT Lincoln Laboratory

999999-9 XYZ 10/22/2011

Automatic avoidance detection algorithm description

Identify the maximum intersected WAF Identify instances of ‘storm avoidance’ (weather avoidance along the departure trajectory path) using the ‘ray’ method Identify avoidance of weather on the departure fix, if the filed departure fix is within 140 km. of the airport

slide-10
SLIDE 10

MIT Lincoln Laboratory

999999-10 XYZ 10/22/2011

Algorithm Description ( intersection )

Identify the maximum intersected WAF

WAF contours

WAF intersection

Maximum intersected WAF

slide-11
SLIDE 11

MIT Lincoln Laboratory

999999-11 XYZ 10/22/2011

Algorithm Description ( ‘ray’ method )

Identify instances of ‘storm avoidance’ (weather avoidance along the departure trajectory path) using the ‘ray’ method Avoidance detected No Avoidance detected

Ray algorithm to identify storm avoidance

Maximum intersected WAF Minimum avoided WAF Maximum avoided WAF

slide-12
SLIDE 12

MIT Lincoln Laboratory

999999-12 XYZ 10/22/2011

Algorithm Description ( departure fix )

Identify avoidance of weather on the departure fix, if the filed departure fix is within 140 km. of the airport

Avoidance of impacted departure fix

Departure fix Flight plan

Minimum avoided WAF Maximum avoided WAF Maximum intersected WAF = 0

trajectory fix

Avoidance detected

trajectory fix

Avoidance detected

slide-13
SLIDE 13

MIT Lincoln Laboratory

999999-13 XYZ 10/22/2011

Algorithm Description ( illustrations of classifications )

Inferred heading Minimum WAF avoided Maximum WAF avoided

Storm avoidance detection Fix avoidance detection

Contour detection Minimum WAF avoided Maximum WAF avoided

Avoidance probability

0.0

0.5

1.0

Weather intersection

Airport Departure fix Airport Departure fix Airport Departure fix

slide-14
SLIDE 14

MIT Lincoln Laboratory

999999-14 XYZ 10/22/2011

Algorithm Description ( validation )

  • Visualizations of 547( NY ) and 257( Chicago ) automated

avoidance classifications were reviewed to validate the algorithm.

  • The error rate was estimated at ~16%.
  • Typical error modes were identified
slide-15
SLIDE 15

MIT Lincoln Laboratory

999999-15 XYZ 10/22/2011

Algorithm Description ( error analysis )

WAF contour fragmentation

Avoidance probability 0.0 0.5 1.0

Contour fragment Intersection ray

Incorrect contour was selected as a cause

  • f avoidance due to

WAF contour fragmentation Overestimating the observed avoidance probability for the lower forecast probability associated with the fragment, while underestimating the observed avoidance probability associated with the higher forecast probability associated with the higher region Incorrect contour was selected as a cause

  • f avoidance due to

WAF contour fragmentation Imaginary closure of fragmented WAF contour

slide-16
SLIDE 16

MIT Lincoln Laboratory

999999-16 XYZ 10/22/2011

Algorithm Description (error analysis)

Small misclassified deviations

Avoidance probability 0.0 0.5 1.0

slide-17
SLIDE 17

MIT Lincoln Laboratory

999999-17 XYZ 10/22/2011

Algorithm Description (error analysis)

Avoidance probability 0.0 0.5 1.0

Misclassified avoidance WAF contour Misclassified avoidance WAF contour

  • a. Trajectory slowed to avoid

departure fix congestion

  • b. Trajectory held to avoid

departure fix congestion

Misclassified congestion avoidance maneuvers

slide-18
SLIDE 18

MIT Lincoln Laboratory

999999-18 XYZ 10/22/2011

Contents

  • Goals and motivations
  • Algorithm description
  • Analysis of results
  • Conclusions and future work
slide-19
SLIDE 19

MIT Lincoln Laboratory

999999-19 XYZ 10/22/2011

Avoidance probability calibration (results)

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Predicted avoidance probability Observed avoidance probability

  • a. Chicago

204 127 88 79 52 75 39 54 294 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

  • b. New York

409 278 254 283 143 180 78 154 642

Calibration of predicted avoidance probabilities

Calibration of departure CWAM from Chicago (a) and New York (b)

slide-20
SLIDE 20

MIT Lincoln Laboratory

999999-20 XYZ 10/22/2011

Avoidance of small, isolated, weak thunderstorms ( results )

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Predicted avoidance probability Observed avoidance probability

  • a. Chicago

204 127 88 79 52 75 39 54 294 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

  • b. New York

409 278 254 283 143 180 78 154 642

Calibration of predicted avoidance probabilities

Avoidance probability 0.0 0.5 1.0

Avoided region Avoided region

  • a. Chicago
  • b. New York
slide-21
SLIDE 21

MIT Lincoln Laboratory

999999-21 XYZ 10/22/2011

Results (Chicago vs. New York )

  • In New York, only 72% of encounters with maximum WAF

probabilities >= 0.9 were avoidances, while that percentage was 88% in Chicago

  • Possible explanation: lower avoidance rate for New York may be

explained by more constrained airspace and stricter avoidance rules in NY airspace

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Predicted avoidance probability Observed avoidance probability

  • a. Chicago

204 127 88 79 52 75 39 54 294 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

  • b. New York

409 278 254 283 143 180 78 154 642

Calibration of predicted avoidance probabilities

slide-22
SLIDE 22

MIT Lincoln Laboratory

999999-22 XYZ 10/22/2011

Results (Avoidance strategy )

  • An avoidance trajectory that avoided the storm core but

encountered less severe weather in the vicinity. OR

  • Avoid all weather and to fly in clear air.
  • a. Trajectory following small storm boundary
  • b. Trajectory following large storm boundary
  • c. Trajectory following ‘best feasible option’ through weather

Avoidance probability 0.0 0.5 1.0

slide-23
SLIDE 23

MIT Lincoln Laboratory

999999-23 XYZ 10/22/2011

5 10 15 20 25 30

Chicago

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Results (Avoidance strategies)

Maximum intersected WAF Percentage of flights

  • ~30%(Chicago)/40%(New York) of flights that avoided WAF of 0.9 avoided all weather
  • ~ 60%/Chicago)/65%(New York) flights avoided WAF with values >= 0.3.

This suggests that pilots will avoid weather near a storm that they would otherwise fly through if that weather were isolated and not associated with the storm

5 10 15 20 25 30 35 40 45

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

New York Maximum intersected WAF Maximum intersected WAF for all flights with maximum avoided WAF = 0.9

slide-24
SLIDE 24

MIT Lincoln Laboratory

999999-24 XYZ 10/22/2011

Contents

  • Goals and motivations
  • Algorithm description
  • Analysis of results
  • Conclusions and future work
slide-25
SLIDE 25

MIT Lincoln Laboratory

999999-25 XYZ 10/22/2011

Conclusions

  • CWAM for departure airspace was evaluated based on data

from 5 days of operations in Chicago and 8 days in New York during the 2010 summer.

  • An automated weather decision classification algorithm was

created

  • The classification error was estimated at ~16
  • The departure CWAM produces a reasonably well-calibrated
  • WAF. But over-warning for high WAFs( esp. for New York ) and

under-warning in low WAFs was detected that matches with RAPT problems

  • Avoidance behavior – the WAF intersections for pilots avoiding

WAF features with avoidance probability of 0.9 – was also analyzed: where possible, pilots seek to avoid all weather impacts, not simply to reduce them to an ‘acceptable’ level

slide-26
SLIDE 26

MIT Lincoln Laboratory

999999-26 XYZ 10/22/2011

Future work

  • Investigating of over/under-warning in WAF in context of RAPT
  • Evaluation of departure CWAM performance based on forecast

weather.

  • Development of a single combined departure / arrival CWAM

for terminal airspace.

  • Enhancements in the automated decision classification
  • algorithm. Application of classification algorithm for verification
  • f various avoidance fields

THANK YOU !