On a Class of Nonparametric Bayesian Autoregressive Models Maria - PowerPoint PPT Presentation

On a Class of Nonparametric Bayesian Autoregressive Models Maria Anna Di Lucca 1 , Alessandra Guglielmi 2 , uller 3 , Fernando A. Quintana 4 Peter M¨ Karolinska Institutet 1 Politecnico di Milano 2 University of Texas, Austin 3 olica de Chile 4 Pontificia Universidad Cat´ ICERM Workshop, Providence, RI, USA, September 17–21, 2012 : slide 1 of 37

Outline Motivation 1 DDP Models 2 The Model 3 Some Previous Work The Model: Continuous Case The Model: Binary Case Data Ilustrations 4 Old Faithful Geyser Data from Multiple Binary Sequences Final Comments 5 : slide 2 of 37

Outline Motivation 1 DDP Models 2 The Model 3 Some Previous Work The Model: Continuous Case The Model: Binary Case Data Ilustrations 4 Old Faithful Geyser Data from Multiple Binary Sequences Final Comments 5 Motivation: slide 3 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Motivation Autoregressive models are very popular. We want to generalize usual assumptions ⇒ parametric case limits the scope and extent of inference. Instead, we want to define a notion of “flexible autoregressive model”. For instance, for order 1 dependence, we would like to replace Y t = β + αY t − 1 + ǫ t by Y t | Y t − 1 = y ∼ F y . Proposal is based on dependent Dirichlet processes (DDP) but method can be extended to other types of random probability measures. Motivation: slide 4 of 37

Outline Motivation 1 DDP Models 2 The Model 3 Some Previous Work The Model: Continuous Case The Model: Binary Case Data Ilustrations 4 Old Faithful Geyser Data from Multiple Binary Sequences Final Comments 5 DDP Models: slide 5 of 37

Dependent Dirichlet Processes (DDP) Given a set of indices { x : x ∈ X } , MacEachern (1999, 2000) proposed to consider ∞ � G x ( · ) = w j ( x ) δ θ j ( x ) ( · ) , x ∈ X . j =1 Barrientos et al. (2012) studied the case w j ( x ) = V j ( x ) � j − 1 i =1 (1 − V i ( x )) , where { V j ( x ) } x ∈ X are i.i.d. stochastic processes (s.p.) such that V j ( x ) ∼ Beta(1 , M x ) for every x ∈ X using copulas! the { θ j ( x ) } x ∈ X are i.i.d. s.p. with θ j ( x ) ∼ G 0 using copulas too! { V j ( x ) } and { θ j ( x ) } vary smoothly with x . DDP Models: slide 6 of 37

Dependent Dirichlet Processes (DDP) Given a set of indices { x : x ∈ X } , MacEachern (1999, 2000) proposed to consider ∞ � G x ( · ) = w j ( x ) δ θ j ( x ) ( · ) , x ∈ X . j =1 Barrientos et al. (2012) studied the case w j ( x ) = V j ( x ) � j − 1 i =1 (1 − V i ( x )) , where { V j ( x ) } x ∈ X are i.i.d. stochastic processes (s.p.) such that V j ( x ) ∼ Beta(1 , M x ) for every x ∈ X using copulas! the { θ j ( x ) } x ∈ X are i.i.d. s.p. with θ j ( x ) ∼ G 0 using copulas too! { V j ( x ) } and { θ j ( x ) } vary smoothly with x . DDP Models: slide 6 of 37

Dependent Dirichlet Processes (DDP) Given a set of indices { x : x ∈ X } , MacEachern (1999, 2000) proposed to consider ∞ � G x ( · ) = w j ( x ) δ θ j ( x ) ( · ) , x ∈ X . j =1 Barrientos et al. (2012) studied the case w j ( x ) = V j ( x ) � j − 1 i =1 (1 − V i ( x )) , where { V j ( x ) } x ∈ X are i.i.d. stochastic processes (s.p.) such that V j ( x ) ∼ Beta(1 , M x ) for every x ∈ X using copulas! the { θ j ( x ) } x ∈ X are i.i.d. s.p. with θ j ( x ) ∼ G 0 using copulas too! { V j ( x ) } and { θ j ( x ) } vary smoothly with x . DDP Models: slide 6 of 37

Dependent Dirichlet Processes (DDP) Given a set of indices { x : x ∈ X } , MacEachern (1999, 2000) proposed to consider ∞ � G x ( · ) = w j ( x ) δ θ j ( x ) ( · ) , x ∈ X . j =1 Barrientos et al. (2012) studied the case w j ( x ) = V j ( x ) � j − 1 i =1 (1 − V i ( x )) , where { V j ( x ) } x ∈ X are i.i.d. stochastic processes (s.p.) such that V j ( x ) ∼ Beta(1 , M x ) for every x ∈ X using copulas! the { θ j ( x ) } x ∈ X are i.i.d. s.p. with θ j ( x ) ∼ G 0 using copulas too! { V j ( x ) } and { θ j ( x ) } vary smoothly with x . DDP Models: slide 6 of 37

Dependent Dirichlet Processes (DDP) Given a set of indices { x : x ∈ X } , MacEachern (1999, 2000) proposed to consider ∞ � G x ( · ) = w j ( x ) δ θ j ( x ) ( · ) , x ∈ X . j =1 Barrientos et al. (2012) studied the case w j ( x ) = V j ( x ) � j − 1 i =1 (1 − V i ( x )) , where { V j ( x ) } x ∈ X are i.i.d. stochastic processes (s.p.) such that V j ( x ) ∼ Beta(1 , M x ) for every x ∈ X using copulas! the { θ j ( x ) } x ∈ X are i.i.d. s.p. with θ j ( x ) ∼ G 0 using copulas too! { V j ( x ) } and { θ j ( x ) } vary smoothly with x . DDP Models: slide 6 of 37

Dependent Dirichlet Processes (DDP) Given a set of indices { x : x ∈ X } , MacEachern (1999, 2000) proposed to consider ∞ � G x ( · ) = w j ( x ) δ θ j ( x ) ( · ) , x ∈ X . j =1 Barrientos et al. (2012) studied the case w j ( x ) = V j ( x ) � j − 1 i =1 (1 − V i ( x )) , where { V j ( x ) } x ∈ X are i.i.d. stochastic processes (s.p.) such that V j ( x ) ∼ Beta(1 , M x ) for every x ∈ X using copulas! the { θ j ( x ) } x ∈ X are i.i.d. s.p. with θ j ( x ) ∼ G 0 using copulas too! { V j ( x ) } and { θ j ( x ) } vary smoothly with x . DDP Models: slide 6 of 37

DDPs (Cont.) Generic form to construct DDPs: use real-valued i.i.d. Gaussian processes { Z j ( x ) } and { U j ( x ) } , j ≥ 1 , with N (0 , 1) marginals, say. For instance, a continuous AR(1) when X = R . define V j ( x ) = B − 1 x (Φ( Z j ( x ))) where B x : CDF for the Beta(1 , M x ) distribution and Φ : N (0 , 1) CDF. define θ j ( x ) = G − 1 0 (Φ( U j ( x ))) . define j − 1 ∞ � � � � G x ( · ) = V j ( x ) (1 − V i ( x )) δ θ j ( x ) ( · ) . j =1 i =1 G x ∼ DP ( M x , G 0 ) for every x ∈ X . DDP Models: slide 7 of 37