Preliminary verifjcation of ensemble precipitation forecast over - - PowerPoint PPT Presentation

▶
preliminary verifjcation of ensemble precipitation
SMART_READER_LITE
LIVE PREVIEW

Preliminary verifjcation of ensemble precipitation forecast over - - PowerPoint PPT Presentation

Mar 08, 2023 142 likes •358 views

Preliminary verifjcation of ensemble precipitation forecast over South America Cristina T oledano (AEMET) Michael Hofg (DWD) Roberto Garcia (CPTEC/INPE) Seyni Salack (WASCAL) S sasasa IVMW 2017 Berlin Project 7 team Predicability

slide-1
SLIDE 1

S

Preliminary verifjcation of ensemble precipitation forecast over South America

sasasa

Cristina T

  • ledano (AEMET)

Michael Hofg (DWD) Roberto Garcia (CPTEC/INPE) Seyni Salack (WASCAL)

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-2
SLIDE 2

Dataset

CPTEC/INPE’s EPS

  • 100 km resolution
  • Forecasts for 15 days
  • 15 members (including control)
  • Initialization time at 12 UTC
  • Output at every 6 hr * 4x = 24 hr (obs)
  • 52 x 67 spatial grid
  • 89 days in rain season (2015-12 to

2016-02)

Observation

  • MERGE (station+satellite)
  • 20 km resolution  100 km (model)
  • 24-hr accumulated precip at 12 UTC

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-3
SLIDE 3

Goal

Verify the precipitation predictability limit in the rainy season over South America.

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-4
SLIDE 4

Goal

Verify the precipitation predictability limit in the rainy season over South America.

Element Spatial domain Temporal domain

IVMW 2017 – Berlin Project 7 team Predicability Limit

Identifjed components are:

slide-5
SLIDE 5

Marginal distribution - Histogram

Obs 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-6
SLIDE 6

Marginal distribution - Histogram

Obs 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h IVMW 2017 – Berlin Project 7 team Predicability Limit

Distributions match

slide-7
SLIDE 7

Joint distributions - Scatterplot

24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h

  • verall

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-8
SLIDE 8

Overforecasting Best-fjt line Regression line

IVMW 2017 – Berlin Project 7 team Predicability Limit

Joint distributions - Scatterplot

Tendenc y to become uncorrel. (no skill)

slide-9
SLIDE 9

Verifjcation – Continuous variables

IVMW 2017 – Berlin Project 7 team Predicability Limit

r decreases

Average of the errors No magnitude Direction: + = overfcst

  • = underfcst

Average of the magnitude of errors

  • No direction

Magnitude of the error No direction Higher are weighted more

difgerence

slide-10
SLIDE 10
  • verfcst

underfcst (best=1)

IVMW 2017 – Berlin Project 7 team Predicability Limit

Frequency bias: whether distribution are similar in the category (Reliability)

Verifjcation – Categorical variables

c a b a BIAS   

slide-11
SLIDE 11

Verifjcation – Categorical variables

(best=1) (best=0) IVMW 2017 – Berlin Project 7 team Predicability Limit More discrimination of ~ rain/no rain

c a a POD   b a b FAR  

slide-12
SLIDE 12

Verifjcation – Ensemble spread

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-13
SLIDE 13

Verifjcation – Ensemble spread

OvFcsting more than unFcsting Some narrowness in spread (U-shape)

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-14
SLIDE 14

Realiability Diagrams – Probability 5mm

24h 72h 120h 192h 264h 360h IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-15
SLIDE 15

Realiability Diagrams – Probability 5mm

IVMW 2017 – Berlin Project 7 team Predicability Limit Reliability = Proximity to diagonal Resolution: Proximity to climatology line probabilities are

  • verestimated

Minimal resolution Sharpeness refers to the spread of the probability distributions

slide-16
SLIDE 16

Verifjcations – Probability 5mm

IVMW 2017 – Berlin Project 7 team Predicability Limit 24h 48h 72h 96h 120h 144h 168h 192h 216h 240h 264h 288h 312h 336h 360h

  • verall
slide-17
SLIDE 17

Verifjcations – Probability 5mm

IVMW 2017 – Berlin Project 7 team Predicability Limit

0.5 = no skill

79% prob of successfully distinguishing 5mm event from non-event

slide-18
SLIDE 18

Performance verifjcation in brief…

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-19
SLIDE 19

Performance verifjcation in brief…

IVMW 2017 – Berlin Project 7 team Predicability Limit 5mm threshold

Overforecast Underforecas t

Perfe ct Bias Hit rate 1-FAR

Critical Success Index

Lead 1 Lead 2 Lead 15

Lead time

slide-20
SLIDE 20

What the verifjcation is showing

  • Mostly over-forecasts.
  • Very sensitive to chosen threshold

(overforecasting weak events, underforecasting strong events)  Is it possible to have a dynamic calibration?

  • The model can discriminate between

events and non-events until very high lead times.

  • But for high thresholds  scores tend to

be the worst.

  • Bad reliability/scores might result from
  • bject shift?! Is it the „double“ penalty

curse???

IVMW 2017 – Berlin Project 7 team Predicability Limit

slide-21
SLIDE 21

Conclusion

What is the predictability limit in the rainy season over South America?

  • No fjnal conclusion can be made, it is just a

preliminary study!

  • Possible reasons for bad scores:
  • The spatial shift  Consider spatial

verifjcation

  • Bad data preparation  review temporal

and spatial matching

IVMW 2017 – Berlin Project 7 team Predicability Limit