Subspace Information Criterion Subspace Information Criterion for - - PowerPoint PPT Presentation

June 11, 2001. SCIA2001 1

Subspace Information Criterion for Image Restoration Subspace Information Criterion for Image Restoration

Department of Computer Science, Tokyo Institute of Technology, Japan. Masashi Sugiyama Hidemitsu Ogawa Mean Squared Error Estimator for Linear Filters Mean Squared Error Estimator for Linear Filters

June 11, 2001. SCIA2001 2

Image Restoration Image Restoration

Restoration Filter We propose a method for determining parameter values appropriately. large small appropriate

Degraded Image Parameter w e.g. Moving-average filter, Regularization filter w:

June 11, 2001. SCIA2001 3

Formulation Formulation

1

H space Hilbert

2

H space Hilbert A

w

X image Original image Restored Noise image Observed n + n Degradatio Filter f g n Af g + = g X f

w w =

ˆ Af

w

f ˆ

w: Parameter

June 11, 2001. SCIA2001 4

Goal of This Talk Goal of This Talk

We want to determine the parameter value w so that Mean Squared Error (MSE) is minimized.

2

ˆ f fw − = MSE

However, it is impossible to directly calculate MSE since f is unknown. We propose an estimator of MSE called the subspace information criterion (SIC), and determine w so that SIC is minimized.

image Restored : ˆ

w

f image Original : f w parameter with

June 11, 2001. SCIA2001 5

Bias Variance

Bias / Variance Decomposition Bias / Variance Decomposition

Bias

2

ˆ f f E E

w −

= MSE

2 2

ˆ ˆ ˆ

w w w

f E f E f f E − + − =

noise

• ver

n Expectatio : E variance Noise :

2

σ

w w

X X

• f

Adjoint :

*

) trace(

* 2 w wX

X σ =

Variance

filter n Restoratio :

w

X image Restored : ˆ

w

f image Original : f

f

w

f ˆ

w

f Eˆ

June 11, 2001. SCIA2001 6

Key Idea for Estimating Bias Key Idea for Estimating Bias

g A fu

1

ˆ

−

= ). ˆ : ˆ f f E f f

u u

= (

• f

estimate Unbiased ! bias! estimating for used is

u

f ˆ

f n A E Af A g A E = + =

− − − 1 1 1

• perator

n Degradatio : A image Degraded : g noise

• ver

n Expectatio : E image Original : f

June 11, 2001. SCIA2001 7

Bias Estimation Bias Estimation

( )

* 2 2

trace ˆ ˆ X X f f

u w

σ − − − = Bias E

1 −

− = A X X

w

E

2 2

, 2 ˆ ˆ n X n X Af X f f

u w

− − − =

2

ˆ f f E w − = Bias

→ g X w

Rough estimate

Bias

g A 1

−

←

Bias Bias = E

f

w

f ˆ

w

f Eˆ

u

f Eˆ =

u

f ˆ

June 11, 2001. SCIA2001 8

Subspace Information Criterion (SIC) Subspace Information Criterion (SIC)

( ) ( )

* 2 * 2 2

trace trace ˆ ˆ

w w u w

X X X X f f σ σ + − − = SIC

MSE SIC E E =

1 −

− = A X X

w

SIC is an unbiased estimator of expected MSE:

variance Noise :

2

σ

Bias estimator Variance

2

ˆ f fw − = MSE

June 11, 2001. SCIA2001 9

Simulation Setting Simulation Setting

Degradation operator A Noise Filter

A

) , (

2

σ N 900

2 =

σ

1 −

′ = A X X filter average

• Moving

: X ′

. . . d i i

) , ( y x n ) , ( ) , ]( [ y x f a y x Af

x

=

) , ( y x f

factor scaling :

x

a

June 11, 2001. SCIA2001 10

Moving-Average Filter Moving-Average Filter

∑

− =

− − =

W W j i ij

j y i x g w y x f

,

) , ( ) , ( ˆ image Restored image Observed f ˆ g ) , ( y x

1 2 + W

ij

w

image Observed

Window

W size Window : Parameter } {

ij

w pattern Weight

Weighted sum

June 11, 2001. SCIA2001 11

Parameter candidates Parameter candidates

• (a) Rhombus (b) Pyramid (c) Gauss

5 , , 1 , K = W size Window } {

ij

w pattern Weight

June 11, 2001. SCIA2001 12

Simulation Results : Lena Simulation Results : Lena

MSE=7046 SIC: Gauss (W=1), MSE=99 OPT: Gauss (W=1)

f ˆ g

June 11, 2001. SCIA2001 13

Simulation Results : Peppers Simulation Results : Peppers

MSE=5572 SIC: Rhombus (W=2), MSE=99 OPT: Rhombus (W=2)

f ˆ g

June 11, 2001. SCIA2001 14

Simulation Results : Girl Simulation Results : Girl

MSE=3217 SIC: Rhombus (W=2), MSE=77 OPT: Rhombus (W=2)

f ˆ g

June 11, 2001. SCIA2001 15

Simulation Setting (2) Simulation Setting (2)

Degradation operator A Noise Filter: Regularization filter

A

) , (

2

σ N 16

2 =

σ

α

X

. . . d i i

) , ( y x n

∑

− =

− =

7 7 15 1

) , ( ) , ]( [

i

y i x f y x Af

) , ( y x f parameter tion Regulariza : α } 10 , , 10 , 10 {

3 4 5

K

− −

from selected

June 11, 2001. SCIA2001 16

Compared Methods Compared Methods

Subspace information criterion (SIC) Mallows’s CL (CL) Leave-one-out cross-validation (CV) Network information criterion (NIC) A Bayesian information criterion (ABIC) Vapnik’s measure (VM)

June 11, 2001. SCIA2001 17

Simulation Results Simulation Results

SIC outperforms other methods !!

June 11, 2001. SCIA2001 18

Conclusions Conclusions

We proposed an unbiased estimator of expected mean squared error (MSE) called subspace information criterion (SIC). Computer simulations showed that

SIC gives a very accurate estimate of

MSE.

Optimal parameter values can be

• btained by SIC.

SIC outperforms other methods.