Statistical Inference in Gaussian Graphical Models Y. Baraud (1) , - PowerPoint PPT Presentation

Statistical Inference in Gaussian Graphical Models Y. Baraud (1) , C. Giraud (1 , 2) , S. Huet (2) , N. Verzelen (3) (1) Universit´ e de Nice, (2) INRA Jouy-en-Josas, (3) Universit´ e Paris Sud Vienna 2008. Christophe GIRAUD Statistical Inference of Gaussian Graphs

Gene - gene regulation network of E. coli Christophe GIRAUD Statistical Inference of Gaussian Graphs

Protein - protein network of S. cerevisiae 1458 proteins (vertices) and their 1948 known interactions (edges) Christophe GIRAUD Statistical Inference of Gaussian Graphs

Inferring gene regulation networks Data: massive transcriptomic data sets produced by microarrays. Differential analysis of data obtained in different conditions: with or without deletion of a gene, with or without stress, etc. Analysis of the conditional dependences in the data (exploits the whole data set). Christophe GIRAUD Statistical Inference of Gaussian Graphs

A few statistical tools Descriptive tools: Kernel methods (supervised learning) Model based tools: Bayesian Networks Gaussian Graphical Models Christophe GIRAUD Statistical Inference of Gaussian Graphs

Gaussian Graphical Models Christophe GIRAUD Statistical Inference of Gaussian Graphs

Gaussian Graphical Models Statistical model: The transcription levels ( X (1) , . . . , X ( p ) ) of the p genes are modeled by a Gaussian law in R p . Graph of the conditional dependences: graph g with g ∼ j between the genes i and j an edge i iff X ( i ) and X ( j ) are not independent given X ( k ) , k � = i , j � � regulation network ← → graph g Christophe GIRAUD Statistical Inference of Gaussian Graphs

The task of the statistician Goal: estimate g from a sample X 1 , . . . , X n . n ≪ p Main difficulty: p ≈ a few 100 to a few 1000 genes n ≈ a few tens New algorithms: based on thresholding or regularization − → many of them have quite disappointing numerical performances (Villers et al. 2008) − → no theoretical results or in an asymptotic framework (with strong hypotheses on the covariance) Christophe GIRAUD Statistical Inference of Gaussian Graphs

Estimation by model selection Christophe GIRAUD Statistical Inference of Gaussian Graphs

Partial correlations Hypothesis: ( X (1) , . . . , X ( p ) ) ∼ N (0 , C ) in R p , with C ≻ 0. � � θ ( j ) Notation: We write θ = for the p × p matrix such that k X ( j ) | X ( k ) , k � = j θ ( j ) k � = j θ ( j ) k X ( k ) . � � = � = 0 and E j Cov ( X ( i ) , X ( j ) | X ( k ) , k � = i , j ) Skeleton of θ : we have θ ( j ) = so i Var ( X ( j ) | X ( k ) , k � = j ) g θ ( j ) � = 0 ⇐ ⇒ i ∼ j i Goal: Estimate θ from a sample X 1 , . . . , X n with quality criterion � � � � MSEP (ˆ � C 1 / 2 (ˆ θ − θ ) � 2 new (ˆ θ − θ ) � 2 � X T θ ) = E = E p × p 1 × p Christophe GIRAUD Statistical Inference of Gaussian Graphs

Partial correlations Hypothesis: ( X (1) , . . . , X ( p ) ) ∼ N (0 , C ) in R p , with C ≻ 0. � � θ ( j ) Notation: We write θ = for the p × p matrix such that k X ( j ) | X ( k ) , k � = j θ ( j ) k � = j θ ( j ) k X ( k ) . � � = � = 0 and E j Cov ( X ( i ) , X ( j ) | X ( k ) , k � = i , j ) Skeleton of θ : we have θ ( j ) = so i Var ( X ( j ) | X ( k ) , k � = j ) g θ ( j ) � = 0 ⇐ ⇒ i ∼ j i Goal: Estimate θ from a sample X 1 , . . . , X n with quality criterion � � � � MSEP (ˆ � C 1 / 2 (ˆ θ − θ ) � 2 new (ˆ θ − θ ) � 2 � X T θ ) = E = E p × p 1 × p Christophe GIRAUD Statistical Inference of Gaussian Graphs

Partial correlations Hypothesis: ( X (1) , . . . , X ( p ) ) ∼ N (0 , C ) in R p , with C ≻ 0. � � θ ( j ) Notation: We write θ = for the p × p matrix such that k X ( j ) | X ( k ) , k � = j θ ( j ) k � = j θ ( j ) k X ( k ) . � � = � = 0 and E j Cov ( X ( i ) , X ( j ) | X ( k ) , k � = i , j ) Skeleton of θ : we have θ ( j ) = so i Var ( X ( j ) | X ( k ) , k � = j ) g θ ( j ) � = 0 ⇐ ⇒ i ∼ j i Goal: Estimate θ from a sample X 1 , . . . , X n with quality criterion � � � � MSEP (ˆ � C 1 / 2 (ˆ θ − θ ) � 2 new (ˆ θ − θ ) � 2 � X T θ ) = E = E p × p 1 × p Christophe GIRAUD Statistical Inference of Gaussian Graphs

Partial correlations Hypothesis: ( X (1) , . . . , X ( p ) ) ∼ N (0 , C ) in R p , with C ≻ 0. � � θ ( j ) Notation: We write θ = for the p × p matrix such that k X ( j ) | X ( k ) , k � = j θ ( j ) k � = j θ ( j ) k X ( k ) . � � = � = 0 and E j Cov ( X ( i ) , X ( j ) | X ( k ) , k � = i , j ) Skeleton of θ : we have θ ( j ) = so i Var ( X ( j ) | X ( k ) , k � = j ) g θ ( j ) � = 0 ⇐ ⇒ i ∼ j i Goal: Estimate θ from a sample X 1 , . . . , X n with quality criterion � � � � MSEP (ˆ � C 1 / 2 (ˆ θ − θ ) � 2 new (ˆ θ − θ ) � 2 � X T θ ) = E = E p × p 1 × p Christophe GIRAUD Statistical Inference of Gaussian Graphs

Estimation strategy Estimation procedure 1 Choose a collection G of candidate graphs e.g. all the graphs with p vertices and degree ≤ D , 2 Associate to each graph g ∈ G an estimator ˆ θ g ˆ � X ( I − A ) � 2 θ g = argmin (empirical MSEP) n × p A ∼ g 3 Select one ˆ θ ˆ g by minimizing a penalized empirical risk with a criterion inspired by that in Baraud et al. Christophe GIRAUD Statistical Inference of Gaussian Graphs

Estimation strategy Estimation procedure 1 Choose a collection G of candidate graphs e.g. all the graphs with p vertices and degree ≤ D , 2 Associate to each graph g ∈ G an estimator ˆ θ g ˆ � X ( I − A ) � 2 θ g = argmin (empirical MSEP) n × p A ∼ g 3 Select one ˆ θ ˆ g by minimizing a penalized empirical risk with a criterion inspired by that in Baraud et al. Christophe GIRAUD Statistical Inference of Gaussian Graphs

Estimation strategy Estimation procedure 1 Choose a collection G of candidate graphs e.g. all the graphs with p vertices and degree ≤ D , 2 Associate to each graph g ∈ G an estimator ˆ θ g ˆ � X ( I − A ) � 2 θ g = argmin (empirical MSEP) n × p A ∼ g 3 Select one ˆ θ ˆ g by minimizing a penalized empirical risk with a criterion inspired by that in Baraud et al. Christophe GIRAUD Statistical Inference of Gaussian Graphs

Estimation strategy Estimation procedure 1 Choose a collection G of candidate graphs e.g. all the graphs with p vertices and degree ≤ D , 2 Associate to each graph g ∈ G an estimator ˆ θ g ˆ � X ( I − A ) � 2 θ g = argmin (empirical MSEP) n × p A ∼ g 3 Select one ˆ θ ˆ g by minimizing a penalized empirical risk with a criterion inspired by that in Baraud et al. Christophe GIRAUD Statistical Inference of Gaussian Graphs

Theorem: risk bound. When deg( G ) = max { deg( g ) , g ∈ G} fulfills n deg( G ) ≤ ρ � 2 , for some ρ < 1 , 1 . 1 + √ log p � 2 then the MSEP of ˆ θ is bounded by θ g ) ∨ � C 1 / 2 ( I − θ ) � 2 � � MSEP (ˆ MSEP (ˆ θ ) ≤ c ρ log( p ) inf + R n n g ∈G Tr( C ) e − κ ρ n � � where R n = O . Christophe GIRAUD Statistical Inference of Gaussian Graphs

Theorem: risk bound. When deg( G ) = max { deg( g ) , g ∈ G} fulfills n deg( G ) ≤ ρ � 2 , for some ρ < 1 , 1 . 1 + √ log p � 2 then the MSEP of ˆ θ is bounded by θ g ) ∨ � C 1 / 2 ( I − θ ) � 2 � � MSEP (ˆ MSEP (ˆ θ ) ≤ c ρ log( p ) inf + R n n g ∈G Tr( C ) e − κ ρ n � � where R n = O . Christophe GIRAUD Statistical Inference of Gaussian Graphs

Theorem: risk bound. When deg( G ) = max { deg( g ) , g ∈ G} fulfills n deg( G ) ≤ ρ � 2 , for some ρ < 1 , 1 . 1 + √ log p � 2 then the MSEP of ˆ θ is bounded by θ g ) ∨ � C 1 / 2 ( I − θ ) � 2 � � MSEP (ˆ MSEP (ˆ θ ) ≤ c ρ log( p ) inf + R n n g ∈G Tr( C ) e − κ ρ n � � where R n = O . Christophe GIRAUD Statistical Inference of Gaussian Graphs

Theorem: risk bound. When deg( G ) = max { deg( g ) , g ∈ G} fulfills n deg( G ) ≤ ρ � 2 , for some ρ < 1 , 1 . 1 + √ log p � 2 then the MSEP of ˆ θ is bounded by θ g ) ∨ � C 1 / 2 ( I − θ ) � 2 � � MSEP (ˆ MSEP (ˆ θ ) ≤ c ρ log( p ) inf + R n n g ∈G Tr( C ) e − κ ρ n � � where R n = O . Christophe GIRAUD Statistical Inference of Gaussian Graphs

Theorem: risk bound. When deg( G ) = max { deg( g ) , g ∈ G} fulfills n deg( G ) ≤ ρ � 2 , for some ρ < 1 , 1 . 1 + √ log p � 2 then the MSEP of ˆ θ is bounded by θ g ) ∨ � C 1 / 2 ( I − θ ) � 2 � � MSEP (ˆ MSEP (ˆ θ ) ≤ c ρ log( p ) inf + R n n g ∈G Tr( C ) e − κ ρ n � � where R n = O . Christophe GIRAUD Statistical Inference of Gaussian Graphs

Theory Condition on the degree Christophe GIRAUD Statistical Inference of Gaussian Graphs

How far can we trust the empirical MSEP? Prediction error: MSEP(ˆ θ ) = E ( � C 1 / 2 ( θ − ˆ θ ) � 2 ) = E ( � C 1 / 2 ( I − ˆ θ ) � 2 ) −� C 1 / 2 ( I − θ ) � 2 Proposition: From empirical to population MSEP Under the previous condition on the degree, we have with large probability 1 (1 − δ ) � C 1 / 2 ( I − ˆ √ n � X ( I − ˆ θ ) � n × p ≤ (1+ δ ) � C 1 / 2 ( I − ˆ θ ) � p × p ≤ θ ) � p × p for all matrices ˆ θ ∈ � g ∈G Θ g . Christophe GIRAUD Statistical Inference of Gaussian Graphs