Subspace Information Criterion Subspace Information Criterion for - PowerPoint PPT Presentation

SCIA2001 June 11, 2001. 1 Subspace Information Criterion Subspace Information Criterion for Image Restoration for Image Restoration Mean Squared Error Estimator Mean Squared Error Estimator for Linear Filters for Linear Filters Department of Computer Science, Tokyo Institute of Technology, Japan. Masashi Sugiyama Hidemitsu Ogawa

SCIA2001 June 11, 2001. 2 Image Restoration Image Restoration Parameter w Restoration Filter e.g. Moving-average filter, Regularization filter Degraded Image small appropriate large w : We propose a method for determining parameter values appropriately.

SCIA2001 June 11, 2001. 3 Formulation Formulation Hilbert space Hilbert space H H 1 2 Degradatio n A f Af = + g Af n Original image Noise ˆ w = f X g + n w Filter ˆ g f w X w w : Parameter Restored image Observed image

SCIA2001 June 11, 2001. 4 Goal of This Talk Goal of This Talk We want to determine the parameter value w so that Mean Squared Error (MSE) is minimized. ˆ Restored image : f 2 w = f w − ˆ with parameter w MSE f Original image f : However, it is impossible to directly calculate MSE since f is unknown. We propose an estimator of MSE called the subspace information criterion (SIC), and determine w so that SIC is minimized.

SCIA2001 June 11, 2001. 5 Bias / Variance Decomposition Bias / Variance Decomposition 2 = ˆ w − MSE E E f f 2 2 = − + − ˆ ˆ ˆ E f f E f E f w w w Variance Bias Expectatio n over noise E : ˆ Restored image f : E ˆ w f f Original image f : Bias w σ 2 Noise variance : ˆ Variance f Restoratio n filter X : w = σ * 2 w trace( X w X ) w * Adjoint of X : X w w

SCIA2001 June 11, 2001. 6 Key Idea for Estimating Bias Key Idea for Estimating Bias ˆ ˆ = Unbiased estimate of ( : ). f f E f f u u − ˆ = 1 f u A g − = − + − = 1 1 1 E A g A Af E A n f Expectatio n over noise E : Original image f : Degraded image g : Degradatio n operator A : ˆ is used for estimating bias! ! f u

SCIA2001 June 11, 2001. 7 Bias Estimation Bias Estimation 2 = E w − ˆ Bias f f − = − 1 X X A 0 w 2 = − − − 2 ˆ ˆ f f 2 X Af , X n X n w u 0 0 0 E E ( ) 2 = − − − σ ˆ ˆ * 2 Bias f f 0 trace X X w u 0 0 = Bias Bias E Bias E ˆ = E ˆ f f f w u ˆ → f X w g − ← ˆ A 1 f g w u Rough estimate

SCIA2001 June 11, 2001. 8 Subspace Information Criterion Subspace Information Criterion (SIC) (SIC) ( ) ( ) 2 = ˆ − ˆ − σ + σ 2 * 2 * SIC f f trace X X trace X X w u 0 0 w w Variance Bias estimator − = − 1 X X A 0 w σ 2 Noise variance : SIC is an unbiased estimator of expected MSE: = SIC MSE E E 2 = f w − ˆ MSE f

SCIA2001 June 11, 2001. 9 Simulation Setting Simulation Setting � Degradation operator A A = ( , ) f x y [ Af ]( x , y ) a f ( x , y ) x scaling factor a : i . d i . . � Noise σ x 2 n ( x , y ) ( 0 , ) N 2 = σ 900 ′ − = 1 � Filter X X A X ′ Moving - average filter :

SCIA2001 June 11, 2001. 10 Moving-Average Filter Moving-Average Filter W ∑ ˆ = − − f ( x , y ) w g ( x i , y j ) ij = − i , j W + g ˆ 2 W 1 f Weighted sum ( x , y ) w ij Window Restored image Observed Observed image image Parameter : Window size W Weight pattern { w } ij

SCIA2001 June 11, 2001. 11 Parameter candidates Parameter candidates = � K Window size W 0 , 1 , , 5 � Weight pattern { w } ij (a) Rhombus (b) Pyramid (c) Gauss

SCIA2001 June 11, 2001. 12 Simulation Results : Lena Simulation Results : Lena g MSE=7046 ˆ f SIC: Gauss ( W =1), MSE=99 OPT: Gauss ( W =1)

SCIA2001 June 11, 2001. 13 Simulation Results : Peppers Simulation Results : Peppers g MSE=5572 ˆ f SIC: Rhombus ( W =2), MSE=99 OPT: Rhombus ( W =2)

SCIA2001 June 11, 2001. 14 Simulation Results : Girl Simulation Results : Girl g MSE=3217 ˆ f SIC: Rhombus ( W =2), MSE=77 OPT: Rhombus ( W =2)

SCIA2001 June 11, 2001. 15 Simulation Setting (2) Simulation Setting (2) � Degradation operator A A ∑ = 7 − ( , ) f x y 1 [ Af ]( x , y ) f ( x i , y ) = − 15 i 7 i . d i . . � Noise σ 2 n ( x , y ) ( 0 , ) N 2 = σ 16 � Filter: Regularization filter X α α Regulariza tion parameter : − − 5 4 3 K selected from { 10 , 10 , , 10 }

SCIA2001 June 11, 2001. 16 Compared Methods Compared Methods � Subspace information criterion (SIC) � Mallows’s C L (C L ) � Leave-one-out cross-validation (CV) � Network information criterion (NIC) � A Bayesian information criterion (ABIC) � Vapnik’s measure (VM)

SCIA2001 June 11, 2001. 17 Simulation Results Simulation Results SIC outperforms other methods !!

SCIA2001 June 11, 2001. 18 Conclusions Conclusions � We proposed an unbiased estimator of expected mean squared error (MSE) called subspace information criterion (SIC). � Computer simulations showed that � SIC gives a very accurate estimate of MSE. � Optimal parameter values can be obtained by SIC. � SIC outperforms other methods.