Nonlinear dimensionality reduction for functional computer code modelling Benjamin Auder CEA - UPMC 24 august 2010 Thesis since 02/2008 PhD supervisors : G´ erard Biau (UPMC) Bertrand Iooss (EDF) Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 1 / 17
Industrial context Framework : life span of reactor vessels. → Several sequences of accidents can occur. Goal = estimate their probabilities of occurrence. Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 2 / 17
Industrial context Framework : life span of reactor vessels. → Several sequences of accidents can occur. Goal = estimate their probabilities of occurrence. Methodology Modelling Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 2 / 17
Industrial context Framework : life span of reactor vessels. → Several sequences of accidents can occur. Goal = estimate their probabilities of occurrence. Methodology Modelling − → Simulation Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 2 / 17
Industrial context Framework : life span of reactor vessels. → Several sequences of accidents can occur. Goal = estimate their probabilities of occurrence. Methodology Modelling − → Simulation − → Computation. → Sensitivity analysis, uncertainty propagation . . . Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 2 / 17
Industrial context Framework : life span of reactor vessels. → Several sequences of accidents can occur. Goal = estimate their probabilities of occurrence. Methodology Modelling − → Simulation − → Computation. → Sensitivity analysis, uncertainty propagation . . . Improve the simulation stage to allow accurate computations Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 2 / 17
❘ ❘ Mathematical formulation n known couples ( x i , y i ) = inputs-outputs of a very slow code : Inputs x i ∈ ❘ p = initial state of physical system ; Outputs y i ∈ C ([ a , b ] , ❘ ) = evolutions of parameters. − → Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 3 / 17
❘ ❘ Mathematical formulation n known couples ( x i , y i ) = inputs-outputs of a very slow code : Inputs x i ∈ ❘ p = initial state of physical system ; Outputs y i ∈ C ([ a , b ] , ❘ ) = evolutions of parameters. Goal = prediction of functional data : y new ≃ ϕ ( x new) . − → Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 3 / 17
Mathematical formulation n known couples ( x i , y i ) = inputs-outputs of a very slow code : Inputs x i ∈ ❘ p = initial state of physical system ; Outputs y i ∈ C ([ a , b ] , ❘ ) = evolutions of parameters. Goal = prediction of functional data : y new ≃ ϕ ( x new) . − → Statistical learning ”regression” ❘ p → C ([ a , b ] , ❘ ) Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 3 / 17
❘ ❘ ❘ ❘ Back to the ”simple” case of y i ∈ ❘ d Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 4 / 17
❘ ❘ ❘ ❘ Back to the ”simple” case of y i ∈ ❘ d dimensionality reduction : 1 r : C ([ a , b ] , ❘ ) → ❘ d (representation) ; Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 4 / 17
❘ ❘ Back to the ”simple” case of y i ∈ ❘ d dimensionality reduction : 1 r : C ([ a , b ] , ❘ ) → ❘ d (representation) ; statistical learning : 2 f : ❘ p (inputs) → ❘ d (reduced outputs) ; Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 4 / 17
Back to the ”simple” case of y i ∈ ❘ d dimensionality reduction : 1 r : C ([ a , b ] , ❘ ) → ❘ d (representation) ; statistical learning : 2 f : ❘ p (inputs) → ❘ d (reduced outputs) ; output space parametrization : 3 R : ❘ d → C ([ a , b ] , ❘ ) (reconstruction). Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 4 / 17
Back to the ”simple” case of y i ∈ ❘ d dimensionality reduction : 1 r : C ([ a , b ] , ❘ ) → ❘ d (representation) ; statistical learning : 2 f : ❘ p (inputs) → ❘ d (reduced outputs) ; output space parametrization : 3 R : ❘ d → C ([ a , b ] , ❘ ) (reconstruction). Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 4 / 17
State of art (More or less) classical methods Functional linear regression : Faraway, 1997 ; Ramsay & Silverman, 2005, . . . Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 5 / 17
State of art (More or less) classical methods Functional linear regression : Faraway, 1997 ; Ramsay & Silverman, 2005, . . . Decomposition on an orthonormal basis, then learning of d -dimensional coefficients : Chiou et al., 2004 ; Govaerts & No¨ el, 2005 ; Bayarri et al., 2007 ; Marrel, 2008 ; Monestiez & Nerini, 2009 Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 5 / 17
State of art (More or less) classical methods Functional linear regression : Faraway, 1997 ; Ramsay & Silverman, 2005, . . . Decomposition on an orthonormal basis, then learning of d -dimensional coefficients : Chiou et al., 2004 ; Govaerts & No¨ el, 2005 ; Bayarri et al., 2007 ; Marrel, 2008 ; Monestiez & Nerini, 2009 (New) goal : minimize the representation dimension d , to simplify the model ; avoid overfitting, while keeping good performances. Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 5 / 17
Riemannian Manifold Learning 1 Applications 2 Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 6 / 17
Riemannian Manifold Learning 1 Applications 2 Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 7 / 17
Local steps RML ≃ preservation of angles and geodesic distances. choose an origin curve y 0 among the y i , (e.g., the mean) ; 1 Fig. : Origin curve y 0 Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 8 / 17
Local steps RML ≃ preservation of angles and geodesic distances. choose an origin curve y 0 among the y i , (e.g., the mean) ; 1 determine a local basis Q 0 = ( e 1 , . . . , e d ) for the tangent space 2 at y 0 (PCA on the neighborhoods curves) ; Fig. : Tangent plane at y 0 + local basis Q 0 Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 8 / 17
Local steps RML ≃ preservation of angles and geodesic distances. choose an origin curve y 0 among the y i , (e.g., the mean) ; 1 determine a local basis Q 0 = ( e 1 , . . . , e d ) for the tangent space 2 at y 0 (PCA on the neighborhoods curves) ; compute the reduced coordinates z i or curves y i ”close” to y 0 by 3 projecting on Q 0 , Fig. : Local coordinates z i on the tangent plane Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 8 / 17
Local steps RML ≃ preservation of angles and geodesic distances. choose an origin curve y 0 among the y i , (e.g., the mean) ; 1 determine a local basis Q 0 = ( e 1 , . . . , e d ) for the tangent space 2 at y 0 (PCA on the neighborhoods curves) ; compute the reduced coordinates z i or curves y i ”close” to y 0 by 3 projecting on Q 0 , then normalize to verify the identity � y − y 0 � = � x − x 0 � . 4 Fig. : Normalization of coordinates z i Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 8 / 17
”far” from y 0 Step 4 : for y far from y 0 (too far for last step to be accurate), y p = predecessor of y on a shortest path from y 0 y i 1 , . . . , y i d = neighbors of y p for which the z i coordinates are known (breadth-first) Fig. : Data y i in dim. D Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 9 / 17
”far” from y 0 Step 4 : for y far from y 0 (too far for last step to be accurate), y p = predecessor of y on a shortest path from y 0 y i 1 , . . . , y i d = neighbors of y p for which the z i coordinates are known (breadth-first) z = r ( y ) computed by.. ..preserving angles as much as possible : � zz p z i j ≃ � yy p y i j ; ..under the normalization constraint � y − y p � = � z − z p � . Fig. : z i = r ( y i ) in dim. d ≪ D Fig. : Data y i in dim. D Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 9 / 17
Examples Fig. : 3D Swissroll, 400 points Fig. : RML Representation Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 10 / 17
Examples Fig. : 3D Swissroll, 400 points Fig. : RML Representation Fig. : 3D Gaussian, 1000 points Fig. : RML Representation Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 10 / 17
Riemannian Manifold Learning 1 Applications 2 Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 11 / 17
Validation step Data : training = { ( x i , y i ) , i = 1 , . . . , n } ; test = { ( x ′ i , y ′ i ) , i = 1 , . . . , m } ; Model predictions : ˆ y ′ i = M ( x ′ i ), i = 1 , . . . , m . Benjamin Auder (CEA - UPMC) Nonlinear dimensionality reduction 24 august 2010 12 / 17
Recommend
More recommend
