Heavy rainfall modeling in high dimensions Philippe Naveau - PowerPoint PPT Presentation

Intro Metric PAM Spectral Conclusions Heavy rainfall modeling in high dimensions Philippe Naveau naveau@lsce.ipsl.fr Laboratoire des Sciences du Climat et l’Environnement (LSCE) Gif-sur-Yvette, France joint work with A. Sabourin, E. Bernard, M. Vrac and O. Mestre FP7-ACQWA, GIS-PEPER, MIRACLE & ANR-McSim, MOPERA 9 novembre 2012

Intro Metric PAM Spectral Conclusions Hourly precipitation for 92 stations, 1992-2011 (Olivier Mestre) ! ! ! 50 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 48 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 46 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 44 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 42 ! − 4 − 2 0 2 4 6 8 Longitudes

Intro Metric PAM Spectral Conclusions Our game plan to handle extremes from this big rainfall dataset Spatial scale Large (country) Local (region) Problem Dimension reduction Spectral density in moderate dimension Data Weekly maxima Heavy hourly rainfall of hourly precipitation excesses Method Clustering algorithms Mixture of for maxima Dirichlet Without imposing a given parametric structure

Intro Metric PAM Spectral Conclusions Clustering of maxima (joint work with E. Bernard, M. Vrac and O. Mestre) Meteo-France subset data Weekly maxima of hourly precipitation 228 points = 19 years x 3 months x 4 weeks at each of the 92 locations Task 1 Clustering 92 grid points into around 10-20 climatologically homogeneous groups wrt spatial dependence

Intro Metric PAM Spectral Conclusions Applying the kmeans algorithm to maxima (15 clusters) PRECIP log(PRECIP) Kmeans ! Kmeans ! ! ! 50 ! 50 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 48 48 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 46 ! ! ! 46 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 44 ! ! ! 44 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 42 42 ! ! Fall Fall − 4 − 2 0 2 4 6 8 − 4 − 2 0 2 4 6 8 Longitudes Longitudes

Intro Metric PAM Spectral Conclusions The scale and shape GEV parameters GEV scale GEV shape 0.01 Fall 0.06 0.11 50 50 0.16 0.21 0.26 0.31 0.35 0.4 0.45 48 48 46 46 1.5 44 44 2 2.5 3 3.5 4 4.5 5 42 42 5.5 6.1 − 4 − 2 0 2 4 6 8 − 4 − 2 0 2 4 6 8 Longitudes Longitudes

Intro Metric PAM Spectral Conclusions Kmeans clusterings Drawbacks Comparing apples and oranges An average of maxima (centroid of a cluster) is not a maximum variances have to be finite Clustering mixed intensity and dependence among maxima Difficult interpretation of clusters Questions How to find an appropriate metric for maxima ? How to create cluster centroids that are maxima ?

Intro Metric PAM Spectral Conclusions A L1 marginal free distance (Cooley, Poncet and N., 2005, N. and al., 2007) d ( x , y ) = 1 2 E | F y ( M ( y )) − F x ( M ( x )) |

Intro Metric PAM Spectral Conclusions A L1 marginal free distance (Cooley, Poncet and N., 2005, N. and al., 2007) d ( x , y ) = 1 2 E | F y ( M ( y )) − F x ( M ( x )) | If M ( x ) and M ( y ) bivariate GEV, then extremal coefficient = 1 + 2 d ( x , y ) 1 − 2 d ( x , y )

Intro Metric PAM Spectral Conclusions Extension for the asymptotically independent case (Ramos and Ledford) Guillou et al, 2012 η -Madogram 1 �� � � � F η ( M ∗ 1 /η ) − F η ( M ∗ 1 /η ν ( η ) = ) 2 E � � X Y � 1 2 E [ | F ( M ∗ X ) − F ( M ∗ = Y ) | ] where F η (resp. F ) is the df of M ∗ 1 /η and M ∗ 1 /η (resp. of X Y M ∗ X and M ∗ Y ) 1 + V η (1 , 1) /V η (1 , ∞ ) − 1 V η (1 , 1) /V η (1 , ∞ ) ν ( η ) = 2

Intro Metric PAM Spectral Conclusions Extension for the asymptotically independent case (Ramos and Ledford) Guillou et al, 2012 Estimation of the η − madogram F X , resp. � � F Y , be the empirical df of M ∗ X i , resp. M ∗ Y i � � N � 1 � � � � F X ( M ∗ X i ) − � F Y ( M ∗ ν ( η ) = Y i ) � � 2 N i =1 � � M ∗ X i , M ∗ be a sample of N bivariate vectors such that Theorem 1. Let Y i � M ∗ � , M ∗ X i Y i b n b n converges in distribution to a bivariate extreme value distribution with an η − extremal function. Then as n → ∞ and N → ∞ � � � d √ ν ( η ) − 1 2 E | F ( M ∗ X ) − F ( M ∗ N Y ) | → [0 , 1] 2 N C ( u, v ) dJ ( u, v ) �

Intro Metric PAM Spectral Conclusions Clusterings Questions How to find an appropriate metric for maxima ? How to create cluster centroids that are maxima ?

Intro Metric PAM Spectral Conclusions Partitioning Around Medoids (PAM) (Kaufman, L. and Rousseeuw, P.J. (1987))

Intro Metric PAM Spectral Conclusions PAM : Choose K initial mediods

Intro Metric PAM Spectral Conclusions PAM : Assign each point to each closest mediod

Intro Metric PAM Spectral Conclusions PAM : Recompute each mediod as the gravity center of each cluster

Intro Metric PAM Spectral Conclusions PAM : continue if a mediod has been moved

Intro Metric PAM Spectral Conclusions PAM : Assign each point to each closest mediod

Intro Metric PAM Spectral Conclusions PAM : Recompute each mediod as the gravity center of each cluster

Intro Metric PAM Spectral Conclusions PAM with K= 15 Fall ! ! ! 50 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 48 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Latitudes ! ! ! ! ! ! ! ! 46 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 44 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 42 ! − 4 − 2 0 2 4 6 8 Longitudes

Intro Metric PAM Spectral Conclusions PAM with K= 20 Fall ! ! 50 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 48 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Latitudes ! ! ! ! ! ! ! ! 46 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 44 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 42 ! − 4 − 2 0 2 4 6 8 Longitudes

Intro Metric PAM Spectral Conclusions ! ! !"#$%&'()*+,-"(.-/0)+ 1234567887+!57992!27:8+ " + b i b i − a i s i = i max( a i , b i ) a i a i � b i , s i ≈ 1 → Well classified a i ∼ b i , s i ≈ 0 → Neutral a i � b i , s i ≈ − 1 → Badly classified

Intro Metric PAM Spectral Conclusions Sil. coeff. for K= 15 Fall ! ! 50 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 48 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 46 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 44 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 42 ! 0.0 median 0.2 0.4 Silhouette width − 4 − 2 0 2 4 6 8 Average silhouette width : 0.09

Intro Metric PAM Spectral Conclusions Summary on clustering of maxima Classical clustering algorithms (kmeans) are not in compliance with EVT Madogram provides a convenient distance that is marginal free and very fast to compute PAM applied with mado preserves maxima and gives interpretable results

Intro Metric PAM Spectral Conclusions Our game plan to handle extremes from this rainfall dataset Spatial scale Large (country) Local (region) Problem Dimension reduction Spectral density in moderate dimension Data Weekly maxima Heavy hourly rainfall of hourly precipitation excesses Method Clustering algorithms Mixture of for maxima Dirichlet

Intro Metric PAM Spectral Conclusions Bayesian Dirichlet mixture model for multivariate excesses (joint work with A. Sabourin) Meteo-France data Wet hourly events at the regional scale (temporally declustered) of moderate dimensions (from 2 to 5) Task 2 Assessing the dependence among rainfall excesses

Intro Metric PAM Spectral Conclusions Focusing on the “Lyon” cluster ! ! ! 50 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 48 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 46 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 44 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 42 ! − 4 − 2 0 2 4 6 8