conditional quantiles with functional covariates an
play

Conditional quantiles with functional covariates: an application to - PowerPoint PPT Presentation

Oct 21, 2022 •15 likes •485 views

Conditional quantiles with functional covariates: an application to Ozone pollution forecasting Herv Cardot, Christophe Crambes & Pascal Sarda Compstat - Prague August 2004 Compstat 2004 - Prague p.1/14 Presentation of the data (1)

  1. Conditional quantiles with functional covariates: an application to Ozone pollution forecasting Hervé Cardot, Christophe Crambes & Pascal Sarda Compstat - Prague August 2004 Compstat 2004 - Prague – p.1/14

  2. Presentation of the data (1) Data (ORAMIP) : Compstat 2004 - Prague – p.2/14

  3. Presentation of the data (1) Data (ORAMIP) : � 9 variables : NO, N 2 , O 3 , WD, WS, . . . (hourly measurements) Compstat 2004 - Prague – p.2/14

  4. Presentation of the data (1) Data (ORAMIP) : � 9 variables : NO, N 2 , O 3 , WD, WS, . . . (hourly measurements) � 6 stations Compstat 2004 - Prague – p.2/14

  5. Presentation of the data (1) Data (ORAMIP) : � 9 variables : NO, N 2 , O 3 , WD, WS, . . . (hourly measurements) � 6 stations � 4 years : 1997 − 2000 ( 15 th May - 15 th Sept) Compstat 2004 - Prague – p.2/14

  6. Presentation of the data (2) 120 100 80 Ozone 60 40 20 0 0 10 20 30 40 50 60 70 hours Compstat 2004 - Prague – p.3/14

  7. Presentation of the data (3) Compstat 2004 - Prague – p.4/14

  8. Presentation of the data (3) � variable of interest : max of O 3 every day: Y = t ( Y 1 , . . . , Y n ) Compstat 2004 - Prague – p.4/14

  9. Presentation of the data (3) � variable of interest : max of O 3 every day: Y = t ( Y 1 , . . . , Y n ) � covariates : NO, N 2 , O 3 , DV or V V : 18h . . . 24h 1h . . . 17h day 0/day 1 . . . . . . . . . . . . X 1 , 1 X 1 , 24 . . . . . . . . . day n − 1 /day n X n, 1 . . . . . . . . . . . . X n, 24 Compstat 2004 - Prague – p.4/14

  10. Presentation of the data (3) � variable of interest : max of O 3 every day: Y = t ( Y 1 , . . . , Y n ) � covariates : NO, N 2 , O 3 , DV or V V : 18h . . . 24h 1h . . . 17h day 0/day 1 . . . . . . . . . . . . X 1 , 1 X 1 , 24 . . . . . . . . . day n − 1 /day n X n, 1 . . . . . . . . . . . . X n, 24 � ( X i , Y i ) i =1 ,...,n couples of random variables with Y i ∈ R and X i ∈ L 2 ( I ) Compstat 2004 - Prague – p.4/14

  11. Presentation of the data (3) � variable of interest : max of O 3 every day: Y = t ( Y 1 , . . . , Y n ) � covariates : NO, N 2 , O 3 , DV or V V : 18h . . . 24h 1h . . . 17h day 0/day 1 . . . . . . . . . . . . X 1 , 1 X 1 , 24 . . . . . . . . . day n − 1 /day n X n, 1 . . . . . . . . . . . . X n, 24 � ( X i , Y i ) i =1 ,...,n couples of random variables with Y i ∈ R and X i ∈ L 2 ( I ) � X i is known in t 1 , . . . , t p ∈ I (equispaced) Compstat 2004 - Prague – p.4/14

  12. Definition of the conditional quantiles Compstat 2004 - Prague – p.5/14

  13. Definition of the conditional quantiles � α ∈ ]0 , 1[ , x ∈ L 2 ( I ) Compstat 2004 - Prague – p.5/14

  14. Definition of the conditional quantiles � α ∈ ]0 , 1[ , x ∈ L 2 ( I ) � α conditional quantile : P ( Y ≤ g α ( X ) | X = x ) = α Compstat 2004 - Prague – p.5/14

  15. Definition of the conditional quantiles � α ∈ ]0 , 1[ , x ∈ L 2 ( I ) � α conditional quantile : P ( Y ≤ g α ( X ) | X = x ) = α � property : g α ( x ) = arg min a ∈ R E ( l α ( Y − a ) | X = x ) with l α ( u ) = | u | + (2 α − 1) u Compstat 2004 - Prague – p.5/14

  16. Presentation of the model Compstat 2004 - Prague – p.6/14

  17. Presentation of the model � model (cf. Koenker and Bassett, 1978) : � g α ( X ) = c + � Ψ α , X � = c + Ψ α ( t ) X ( t ) dt I Compstat 2004 - Prague – p.6/14

  18. Presentation of the model � model (cf. Koenker and Bassett, 1978) : � g α ( X ) = c + � Ψ α , X � = c + Ψ α ( t ) X ( t ) dt I � we want to estimate the function Ψ α ∈ L 2 ( I ) : spline estimation Compstat 2004 - Prague – p.6/14

  19. Spline estimation of Ψ α Compstat 2004 - Prague – p.7/14

  20. Spline estimation of Ψ α k ∈ N ⋆ , q ∈ N Compstat 2004 - Prague – p.7/14

  21. Spline estimation of Ψ α k ∈ N ⋆ , q ∈ N interval I I j I 1 I k k sub−intervals Compstat 2004 - Prague – p.7/14

  22. Spline estimation of Ψ α k ∈ N ⋆ , q ∈ N t ( B 1 , . . . , B k + q ) B -splines basis B k , q = Compstat 2004 - Prague – p.7/14

  23. Spline estimation of Ψ α k ∈ N ⋆ , q ∈ N t ( B 1 , . . . , B k + q ) B -splines basis B k , q = k + q � t B k , q � � estimator : � Ψ α = θ = θ j B j j =1 Compstat 2004 - Prague – p.7/14

  24. Spline estimation of Ψ α k ∈ N ⋆ , q ∈ N t ( B 1 , . . . , B k + q ) B -splines basis B k , q = k + q � t B k , q � � estimator : � Ψ α = θ = θ j B j j =1 Compstat 2004 - Prague – p.7/14

  25. c and � Determination of � θ Compstat 2004 - Prague – p.8/14

  26. c and � Determination of � θ � � θ and � c solution of the minimisation problem : � 1 n � l α ( Y i − c − � t B k , q θ , X i � ) + ρ � ( t B k , q θ ) ( m ) � 2 � min n θ ∈ R k + q i =1 Compstat 2004 - Prague – p.8/14

  27. c and � Determination of � θ � � θ and � c solution of the minimisation problem : � 1 n � l α ( Y i − c − � t B k , q θ , X i � ) + ρ � ( t B k , q θ ) ( m ) � 2 � min n θ ∈ R k + q i =1 empirical version of E ( l α ( Y − c − � s, X � )) Compstat 2004 - Prague – p.8/14

  28. c and � Determination of � θ � � θ and � c solution of the minimisation problem : � 1 n � l α ( Y i − c − � t B k , q θ , X i � ) + ρ � ( t B k , q θ ) ( m ) � 2 � min n θ ∈ R k + q i =1 penalization Compstat 2004 - Prague – p.8/14

  29. c and � Determination of � θ � � θ and � c solution of the minimisation problem : � 1 n � l α ( Y i − c − � t B k , q θ , X i � ) + ρ � ( t B k , q θ ) ( m ) � 2 � min n θ ∈ R k + q i =1 � no explicit solution Compstat 2004 - Prague – p.8/14

  30. c and � Determination of � θ � � θ and � c solution of the minimisation problem : � 1 n � l α ( Y i − c − � t B k , q θ , X i � ) + ρ � ( t B k , q θ ) ( m ) � 2 � min n θ ∈ R k + q i =1 � no explicit solution � algorithm : Iterative Reweighted Least Squares Compstat 2004 - Prague – p.8/14

  31. Multiple conditional quantiles Compstat 2004 - Prague – p.9/14

  32. Multiple conditional quantiles � v covariates X 1 , . . . , X v Compstat 2004 - Prague – p.9/14

  33. Multiple conditional quantiles � v covariates X 1 , . . . , X v � model : g α ( X 1 , . . . , X v ) � � Ψ 1 α ( t ) X 1 ( t ) dt + . . . + Ψ v α ( t ) X v ( t ) dt = c + I I Compstat 2004 - Prague – p.9/14

  34. Multiple conditional quantiles � v covariates X 1 , . . . , X v � model : g α ( X 1 , . . . , X v ) � � Ψ 1 α ( t ) X 1 ( t ) dt + . . . + Ψ v α ( t ) X v ( t ) dt = c + I I � algorithm : backfitting + Iterative Reweighted Least Squares Compstat 2004 - Prague – p.9/14

  35. Application to the pollution data Compstat 2004 - Prague – p.10/14

  36. Application to the pollution data � learning sample : ( X l i , Y l i ) i =1 ,...,n learn � test sample : ( X t i , Y t i ) i =1 ,...,n test Compstat 2004 - Prague – p.10/14

  37. Application to the pollution data � learning sample : ( X l i , Y l i ) i =1 ,...,n learn � test sample : ( X t i , Y t i ) i =1 ,...,n test � number of knots : k = 8 (equispaced) � degree of splines functions : q = 3 � order of derivation in the penalization : m = 2 Compstat 2004 - Prague – p.10/14

  38. Application to the pollution data � learning sample : ( X l i , Y l i ) i =1 ,...,n learn � test sample : ( X t i , Y t i ) i =1 ,...,n test � number of knots : k = 8 (equispaced) � degree of splines functions : q = 3 � order of derivation in the penalization : m = 2 � choice of ρ : Generalized Cross Validation Compstat 2004 - Prague – p.10/14

  39. Quality criteria of the models Compstat 2004 - Prague – p.11/14

  40. Quality criteria of the models � n t i =1 ( Y t i − � 1 Y t i ) 2 n t C 1 = � n t 1 i =1 ( Y t i − Y l ) 2 n t Compstat 2004 - Prague – p.11/14

  41. Quality criteria of the models � n t i =1 ( Y t i − � 1 Y t i ) 2 n t C 1 = � n t 1 i =1 ( Y t i − Y l ) 2 n t n t � C 2 = 1 | Y t i − � Y t i | n t i =1 Compstat 2004 - Prague – p.11/14

  42. Quality criteria of the models � n t i =1 ( Y t i − � 1 Y t i ) 2 n t C 1 = � n t 1 i =1 ( Y t i − Y l ) 2 n t n t � C 2 = 1 | Y t i − � Y t i | n t i =1 � n t i =1 l α ( Y t i − � 1 Y t i ) n t C 3 = � n t 1 i =1 l α ( Y t i − q α ( Y l )) n t Compstat 2004 - Prague – p.11/14

  43. Results (conditional median) Compstat 2004 - Prague – p.12/14

  44. Results (conditional median) Models Variables C 1 C 2 C 3 N2 0 . 814 16 . 916 0 . 906 1 covariate O3 0.414 12.246 0.656 WS 0 . 802 16 . 836 0 . 902 O3, NO 0 . 413 11 . 997 0 . 643 2 covariates 0 . 413 11 . 880 0 . 637 O3, N2 O3, WS 0 . 414 12 . 004 0 . 635 O3, NO, N2 0 . 412 12 . 127 0 . 644 3 covariates O3, N2, WD 0 . 409 12 . 004 0 . 645 O3, N2, WS 0 . 410 11 . 997 0 . 642 4 covariates O3, NO, N2, WS 0.400 11.718 0.634 5 covariates O3, NO, N2, WD, WS 0 . 401 11 . 750 0 . 639 Compstat 2004 - Prague – p.12/14

  45. Forecasting (conditional median) Compstat 2004 - Prague – p.13/14

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Jason Roberts, October 2015 Topics for this session Why use covariates other than x and y?

Jason Roberts, October 2015 Topics for this session Why use covariates other than x and y?

Jason Roberts, October 2015 Topics for this session Why use covariates other than x and y? What other covariates are there? Dynamic spatial covariates: how hard can it be? Covariates for our sperm whale model Man any im images s

1.29k views • 89 slides
Discovering conditional Functional Dependencies We n f e i F a n , F l o r i s G e e r t s ,

Discovering conditional Functional Dependencies We n f e i F a n , F l o r i s G e e r t s ,

Discovering conditional Functional Dependencies We n f e i F a n , F l o r i s G e e r t s , J i a n z h o n g L i , a n d M i n g X i o n g I C D E 2 0 0 9 Amira Ghenai Outline Introduction and Motivation Contributions of the

552 views • 24 slides
Time-dependent covariates In many situations it is useful to consider covariates that change over

Time-dependent covariates In many situations it is useful to consider covariates that change over

Time-dependent covariates In many situations it is useful to consider covariates that change over time. These are called time-dependent covariates. Such are of two kinds: 1. Internal variables These are related to each patient and are

325 views • 15 slides
Handling Covariates in the Design Rosenberger of Clinical Trials I. Introduction Covariates and

Handling Covariates in the Design Rosenberger of Clinical Trials I. Introduction Covariates and

Handling Covariates in Clinical Trials W. F. Handling Covariates in the Design Rosenberger of Clinical Trials I. Introduction Covariates and randomized phase III trials Two approaches for handling William F. Rosenberger covariates

664 views • 24 slides
Biased Quantiles Graham Cormode Flip Korn cormode@bell-labs.com flip@research.att.com S.

Biased Quantiles Graham Cormode Flip Korn cormode@bell-labs.com flip@research.att.com S.

Biased Quantiles Graham Cormode Flip Korn cormode@bell-labs.com flip@research.att.com S. Muthukrishnan Divesh Srivastava muthu@cs.rutgers.edu divesh@research.att.com Quantiles Quantiles summarize data distribution concisely. Given N items,

428 views • 23 slides
Effective computation of biased quantiles over data streams Graham Cormode Flip Korn

Effective computation of biased quantiles over data streams Graham Cormode Flip Korn

Effective computation of biased quantiles over data streams Graham Cormode Flip Korn cormode@bell-labs.com flip@research.att.com S. Muthukrishnan Divesh Srivastava muthu@cs.rutgers.edu divesh@research.att.com Quantiles Quantiles

406 views • 23 slides
Deterministic Algorithms for Biased Quantiles Graham Cormode Flip Korn cormode@bell-labs.com

Deterministic Algorithms for Biased Quantiles Graham Cormode Flip Korn cormode@bell-labs.com

Deterministic Algorithms for Biased Quantiles Graham Cormode Flip Korn cormode@bell-labs.com flip@research.att.com S. Muthukrishnan Divesh Srivastava muthu@cs.rutgers.edu divesh@research.att.com Quantiles Quantiles summarize data

247 views • 20 slides
Fluctuations of the empirical quantiles of independent Brownian motions Jason Swanson Department

Fluctuations of the empirical quantiles of independent Brownian motions Jason Swanson Department

Fluctuations of the empirical quantiles of independent Brownian motions Jason Swanson Department of Mathematics University of Central Florida October 9, 2008 -quantiles a probability measure ( ) = ( ) ( , ] x x

858 views • 34 slides
Stat 5101 Lecture Slides: Deck 4 Quantiles and Best Prediction Charles J. Geyer School of

Stat 5101 Lecture Slides: Deck 4 Quantiles and Best Prediction Charles J. Geyer School of

Stat 5101 Lecture Slides: Deck 4 Quantiles and Best Prediction Charles J. Geyer School of Statistics University of Minnesota 1 Quantiles If X is a random variable and 0 < q < 1, then the q -th quantile of X (or of the distribution of X )

747 views • 17 slides
Improving the efficiency of individualized designs for the covariates mixed logit choice model by

Improving the efficiency of individualized designs for the covariates mixed logit choice model by

Outline Introduction Analysis of the mixed logit choice model with Improving the efficiency of individualized designs for the covariates mixed logit choice model by including covariates Including covariates in experimental design for the

273 views • 24 slides
15. The Conditional 15.1 The conditional: Formation and uses 15.2 Mise en pratique 15.1 The

15. The Conditional 15.1 The conditional: Formation and uses 15.2 Mise en pratique 15.1 The

15. The Conditional 15.1 The conditional: Formation and uses 15.2 Mise en pratique 15.1 The Conditional: Formation and Uses Mme Legendre is talking to a friend about her hopes and wishes for the future: Un jour, jaimerais changer de

191 views • 15 slides
Motivation Computation and Aggregation of Quantiles Application at Lucent Technologies: from

Motivation Computation and Aggregation of Quantiles Application at Lucent Technologies: from

Motivation Computation and Aggregation of Quantiles Application at Lucent Technologies: from Data Streams software to monitor distributed IP-based services. John Chambers, David James, Goal: characterize various metrics (e.g. e- Diane

392 views • 3 slides
P( ) 1 conditional probability where P(F) > 0 Conditional probability of E given F:

P( ) 1 conditional probability where P(F) > 0 Conditional probability of E given F:

Conditional Probability P( ) 1 conditional probability where P(F) > 0 Conditional probability of E given F: probability that E occurs given that F has occurred. Ehf F FE Conditioning on F Written as P(E|F) Ser Means P(E,

731 views • 35 slides
Conditional Heteroscedasticity (CH) So far, our models are for the conditional mean . For

Conditional Heteroscedasticity (CH) So far, our models are for the conditional mean . For

Conditional Heteroscedasticity (CH) So far, our models are for the conditional mean . For instance, the Gaussian AR(1) model + t y t = y t 1 may be written: , 2 Conditionally on y t 1 , y

411 views • 19 slides
Markov random fields 2. conditional specifications 3. conditional auto-regression Rasmus

Markov random fields 2. conditional specifications 3. conditional auto-regression Rasmus

Outline: 1. specification of joint distributions Markov random fields 2. conditional specifications 3. conditional auto-regression Rasmus Waagepetersen 4. Brooks factorization, 5. conditional independence and graphs 6. Hammersley-Clifford

202 views • 9 slides
Conditional Probability & Independence Conditional Probabilities Question : How should we

Conditional Probability & Independence Conditional Probabilities Question : How should we

Conditional Probability & Independence Conditional Probabilities Question : How should we modify P ( E ) if we learn that event F has occurred? Definition : the conditional probability of E given F is P ( E | F ) = P ( E \ F ) , for P

593 views • 24 slides
Project Report Study sites: Haryana (Palwal district) and Orissa (Khurda district), India 1

Project Report Study sites: Haryana (Palwal district) and Orissa (Khurda district), India 1

[Type text] Project: Assessment of User-Friendly Product Presentation for Amoxicillin Dispersible Tablets in the Treatment of Childhood Pneumonia in Low-Resource Settings Project Report Study sites: Haryana (Palwal district) and Orissa (Khurda

347 views • 16 slides
A Production-Based Economic Explanation for the Gross Profitability Premium Discussion by

A Production-Based Economic Explanation for the Gross Profitability Premium Discussion by

A Production-Based Economic Explanation for the Gross Profitability Premium Discussion by Jessica A. Wachter September 27, 2019 Jessica A. Wachter Profitability Premium Discussion 1 What is the profitability premium? First define

515 views • 23 slides
Alpine Cultural Heritage Protection: approaches and techniques CHEERS project: Insights on WP T3

Alpine Cultural Heritage Protection: approaches and techniques CHEERS project: Insights on WP T3

Alpine Cultural Heritage Protection: approaches and techniques CHEERS project: Insights on WP T3 Christian Iasio, Carola Mirgon (BRGM) Philippe Meresse, Ronan de Saint German (Entente Valabre) Franck Bourier, Philomene Favier (IRSTEA) Partner

266 views • 22 slides
Associate Professor University of Cincinnati Linda Levin, PhD Associate Professor University of

Associate Professor University of Cincinnati Linda Levin, PhD Associate Professor University of

Linda Levin, PhD Associate Professor University of Cincinnati Linda Levin, PhD Associate Professor University of Cincinnati Purpose of Analyses Reconstruct occupational exposure levels Estimate the impact of exposure on workers

1.01k views • 33 slides
Track sponsored by Understanding the Omnichannel Shift What Forces are Shaping Todays

Track sponsored by Understanding the Omnichannel Shift What Forces are Shaping Todays

Track sponsored by Understanding the Omnichannel Shift What Forces are Shaping Todays Digitally Integrated Shopping Experience E-commerce is rapidly evolving into New Retail. The boundary between offline and online commerce disappears

503 views • 32 slides
TAM Air An Eight Channel Isothermal Heat Flow Calorimeter for TAM Air Cement Cement / Concrete

TAM Air An Eight Channel Isothermal Heat Flow Calorimeter for TAM Air Cement Cement / Concrete

TAM Air An Eight Channel Isothermal Heat Flow Calorimeter for TAM Air Cement Cement / Concrete Research and Production Control use in the mW range Dr. Thomas Lemke C3 Prozess- und Analysentechnik GmbH Outline Isothermal calorimetry

863 views • 41 slides
O n t h e u s e f u l n e s s o f i m p r e c i s e B a y e s i a

O n t h e u s e f u l n e s s o f i m p r e c i s e B a y e s i a

O n t h e u s e f u l n e s s o f i m p r e c i s e B a y e s i a n i s m i n c h e m i c a l k i n e t i c s D r . Ma r c F i s c h e r A s s i s t a n t p r o f e s

462 views • 17 slides
So Softw tware are En Engin gineering eering By Sanjay Shamsundar Manvatkar Analysis &

So Softw tware are En Engin gineering eering By Sanjay Shamsundar Manvatkar Analysis &

So Softw tware are En Engin gineering eering By Sanjay Shamsundar Manvatkar Analysis & Design Tools Decision tree & decision Table DFD (Data Flow Diagram) DD ( Data Dictionary) 1) Element of DD 2) Advantages &

293 views • 27 slides

More recommend