Truncation Error in Image Interpolation Loc Simon SampTA 2013 - - - PowerPoint PPT Presentation

▶
truncation error in image interpolation
SMART_READER_LITE
LIVE PREVIEW

Truncation Error in Image Interpolation Loc Simon SampTA 2013 - - - PowerPoint PPT Presentation

Mar 20, 2023 212 likes •738 views

Truncation Error in Image Interpolation Loc Simon SampTA 2013 - Bremen 1 Collaborator Jean-Michel Morel 2 Truncation error: What is that? X k s X t s 3 Truncation error: What is that? X X t := X k sinc( t k ) k Z 2 4

slide-1
SLIDE 1

Truncation Error in Image Interpolation

Loïc Simon

SampTA 2013 - Bremen

1

slide-2
SLIDE 2

Collaborator

Jean-Michel Morel

2

slide-3
SLIDE 3

Truncation error:

What is that?

Xk’s Xt’s

3

slide-4
SLIDE 4

Truncation error:

What is that?

Xt := X

k∈Z2

Xksinc(t − k)

4

slide-5
SLIDE 5

Truncation error:

What is that?

?

5

slide-6
SLIDE 6

Context

  • Motivations
  • Assumptions
  • Goal
  • Related work

6

slide-7
SLIDE 7

Motivations

  • Image registration
  • optical flow
  • stereopsis
  • super-resolution
  • sub-pixel accuracy

7

slide-8
SLIDE 8

Motivations

  • Image registration
  • optical flow
  • stereopsis
  • super-resolution
  • sub-pixel accuracy
  • error ~ quantization

8

slide-9
SLIDE 9

Assumptions

9

slide-10
SLIDE 10

Assumptions

10

slide-11
SLIDE 11

t Xt K −K

Assumptions

  • a 1d random process
  • observed on
  • weakly stationary
  • no aliasing

µ, dΨX(ω) Xt (t ∈ R) k ∈ {−K, . . . , K} Xt := X

k∈Z

Xksinc(t − k)

11

slide-12
SLIDE 12

Goal

  • Linear shift-invariant
  • Practical bounds on

RMSE[ ˜ Xt] := r E h ( ˜ Xt − Xt)2 i ˜ Xt := X

k≤K

Xkh(t − k)

12

slide-13
SLIDE 13

Goal

  • Linear shift-invariant
  • Practical bounds on

˜ Xt := X

k≤K

Xkh(t − k) RMSE[ ˜ Xt] := r E h ( ˜ Xt − Xt)2 i

⇢ h(t) = sinc(t) h(t) = sincdK(t) ➡ DFT interpolation ➡ Sinc interpolation

13

slide-14
SLIDE 14

Related Work

Jagerman 1966 Yao & Thomas 1966 Campbell 1968 Brown 1969 Xu & Huang & Li 2009

Truncation Error

Strang & Fix 1971 Blu & Unser 1999 Condat & al. 2005

Approximation

Moisan 2011

Other

Jerri 1977

14

slide-15
SLIDE 15

Related Work

Jagerman 1966 Yao & Thomas 1966 Campbell 1968 Brown 1969 Xu & Huang & Li 2009

Truncation Error

Strang & Fix 1971 Blu & Unser 1999 Condat & al. 2005

Approximation

Moisan 2011

Other

Jerri 1977 ➡ oversampled case ➡ sinc only

15

slide-16
SLIDE 16

Related Work

Jagerman 1966 Yao & Thomas 1966 Campbell 1968 Brown 1969 Xu & Huang & Li 2009

Truncation Error

Strang & Fix 1971 Blu & Unser 1999 Condat & al. 2005

Approximation

Moisan 2011

Other

Jerri 1977 ➡ K = ∞

16

slide-17
SLIDE 17

Related Work

Jagerman 1966 Yao & Thomas 1966 Campbell 1968 Brown 1969 Xu & Huang & Li 2009

Truncation Error

Strang & Fix 1971 Blu & Unser 1999 Condat & al. 2005

Approximation

Moisan 2011

Other

Jerri 1977

17

slide-18
SLIDE 18

Rest of the talk

  • Theoretical bounds
  • Experimental results
  • Discussion & conclusion

18

slide-19
SLIDE 19

A bit of intuition...

t δ(t)

δ(t)

19

slide-20
SLIDE 20

Theoretical bounds

MSE[ ˜ X](t) = sin2(πt) π2 × B B B B B B B @ µ2O ⇣

1 δ(t)2

⌘ + σ02

αO

⇣

1 δ(t)2

⌘ + σ2

αO

⇣

1 δ(t)

⌘ 1 C C C C C C C A

20

slide-21
SLIDE 21

Theoretical bounds

MSE[ ˜ X](t) = sin2(πt) π2 × B B B B B B B @ µ2O ⇣

1 δ(t)2

⌘ + σ02

αO

⇣

1 δ(t)2

⌘ + σ2

αO

⇣

1 δ(t)

⌘ 1 C C C C C C C A

21

slide-22
SLIDE 22

Theoretical bounds

MSE[ ˜ X](t) = sin2(πt) π2 × B B B B B B B @ 0µ2O ⇣

1 δ(t)2

⌘ + 2σ02

αO

⇣

1 δ(t)2

⌘ + 2σ2

αO

⇣

1 δ(t)

⌘ 1 C C C C C C C A

➡ DFT modifications

22

slide-23
SLIDE 23

Spectral representation

➡ Spectral component ➡ Average component

MSE[ ˜ X](t) = µ2

  • 1 −

X

|k|≤K

h(t − k)

  • 2

| {z }

MSE[µ](t)

+ 1 2π Z

  • eiωt −

X

|k|≤K

eiωkh(t − k)

  • 2

dΨX(ω) | {z }

MSE[dΨX](t)

23

slide-24
SLIDE 24

Spectral representation

MSE[ ˜ X](t) = µ2

  • 1 −

X

|k|≤K

h(t − k)

  • 2

| {z }

MSE[µ](t)

+ 1 2π Z

  • eiωt −

X

|k|≤K

eiωkh(t − k)

  • 2

dΨX(ω) | {z }

MSE[dΨX](t)

➡ Aliasing is not forbidden

24

slide-25
SLIDE 25

Spectral representation

➡ Under no aliasing condition

MSE[ ˜ X](t) = µ2

  • X

k∈Z

sinc(t − k) − X

|k|≤K

h(t − k)

  • 2

| {z }

MSE[µ](t)

+ 1 2π Z

|ω|≤π

  • X

k∈Z

eiωksinc(t − k) − X

|k|≤K

eiωkh(t − k)

  • 2

dΨX(ω) | {z }

MSE[dΨX](t)

25

slide-26
SLIDE 26

Spectral representation

➡ Under no aliasing condition

MSE[ ˜ X](t) = µ2

  • X

k∈Z

sinc(t − k) − X

|k|≤K

sinc(t − k)

  • 2

| {z }

MSE[µ](t)

+ 1 2π Z

|ω|≤π

  • X

k∈Z

eiωksinc(t − k) − X

|k|≤K

eiωksinc(t − k)

  • 2

dΨX(ω) | {z }

MSE[dΨX](t)

➡ Sinc

26

slide-27
SLIDE 27

Average component

➡ Gibbs phenomenon MSE[µ](t) = sin2(πt) π2 µ2O ✓ 1 δ(t)2 ◆

27

slide-28
SLIDE 28

Average component

➡ DFT MSE[µ](t) = 0

28

slide-29
SLIDE 29

Spectral component

0.50 0.50 10 10 20 20 30 30 40 40 42.8dB 42.8dB 15.2dB 15.2dB Spectrum (dB) Spectrum (dB) 0.50 0.50 10 10 20 20 30 30 40 40 50 50 60 60 54.2dB 54.2dB 23.0dB 23.0dB Spectrum (dB) Spectrum (dB)

0.50 0.50 10 10 20 20 30 30 40 40 36.5dB 36.5dB 1.7dB 1.7dB Spectrum (dB) Spectrum (dB)

0.50 0.50 10 10 20 20 30 30 40 40 41.6dB 41.6dB 9.8dB 9.8dB Spectrum (dB) Spectrum (dB)

29

slide-30
SLIDE 30

Spectral decomposition

➡ spectrum ≤ oversampled + white-noise

ω απ σ2

α

π σ2

α |ω|πdω

dΨ0

α(ω)

ψα(ω)dω dΨX(ω)

30

slide-31
SLIDE 31

Oversampled case

supp(dΨ0

α) ⊂ {|ω| ≤ απ}

= ⇒ MSE[dΨ0

α](t) =sin2(πt)

π2 σ02

α O

✓ 1 δ(t)2 ◆

where,

σ02

α = 1

π Z

|ω|απ

1 1 + cos(ω)dΨ0

α(ω)

31

slide-32
SLIDE 32

Oversampled case

where,

σ02

α = 1

π Z

|ω|απ

1 1 + cos(ω)dΨ0

α(ω)

supp(dΨ0

α) ⊂ {|ω| ≤ απ}

= ⇒ ⇢ MSE[dΨ0

α](t)

MSE[µ](t)

  • =sin2(πt)

π2 ⇢ σ02

α

µ2

  • O

✓ 1 δ(t)2 ◆

32

slide-33
SLIDE 33

White-noise

➡ Slow decay

ω απ σ2

α

π σ2

α |ω|πdω

dΨ0

α(ω)

ψα(ω)dω dΨX(ω)

dΨ(ω) = σ2

α |ω|≤πdω

= ⇒ MSE[dΨ](t) =sin2(πt) π2 σ2

αO

✓ 1 δ(t) ◆

33

slide-34
SLIDE 34

Recap

MSE[ ˜ X](t) = sin2(πt) π2 × B B B B B B B @ 0µ2O ⇣

1 δ(t)2

⌘ + 2σ02

αO

⇣

1 δ(t)2

⌘ + 2σ2

αO

⇣

1 δ(t)

⌘ 1 C C C C C C C A

➡ DFT modifications

34

slide-35
SLIDE 35

Experimental results

  • Bound validity
  • Bound tightness
  • Order of magnitude

➡ online demo: IPOL · Image Processing On Line

35

slide-36
SLIDE 36

Experimental results

  • Bound validity
  • Bound tightness
  • Order of magnitude

➡ online demo: IPOL · Image Processing On Line

36

slide-37
SLIDE 37

Experimental results

  • Bound validity
  • Bound tightness
  • Order of magnitude

➡ online demo: IPOL · Image Processing On Line

37

slide-38
SLIDE 38

Experimental results

  • Bound validity
  • Bound tightness
  • Order of magnitude

➡ online demo: IPOL · Image Processing On Line

38

slide-39
SLIDE 39

Bound tightness

➡ Smooth image

Sinc Sinc w/o µ DFT

➡ q E[quant2] = 0.3

39

slide-40
SLIDE 40

Bound tightness

➡ Simulated white-noise

Sinc Sinc w/o µ DFT

➡ q E[quant2] = 0.3

40

slide-41
SLIDE 41

Bound tightness

➡ Textured image

Sinc Sinc w/o µ DFT

➡ q E[quant2] = 0.3

41

slide-42
SLIDE 42

Other kernels?

  • Bound validity
  • Bound tightness
  • Order of magnitude

➡ online demo: IPOL · Image Processing On Line

42

slide-43
SLIDE 43

Other kernels?

Sinc w/o µ

43

slide-44
SLIDE 44

Other kernels?

Sinc + accel

44

slide-45
SLIDE 45

Other kernels?

Bilinear

45

slide-46
SLIDE 46

Other kernels?

Bicubic

46

slide-47
SLIDE 47

Other kernels?

B-Spline 3

47

slide-48
SLIDE 48

Other kernels?

B-Spline 11

48

slide-49
SLIDE 49

Conclusion

  • Textures are nasty
  • Aliasing is not the worst thing in life
  • Is there a hope for image interpolation?

49

slide-50
SLIDE 50

Empirical estimate

  • ...

50

slide-51
SLIDE 51

Textures

  • What’s special about images?

51