A Fast Spatial Patch Blending Algorithm for Artefact Reduction in - - PowerPoint PPT Presentation

▶
a fast spatial patch blending algorithm for artefact
SMART_READER_LITE
LIVE PREVIEW

A Fast Spatial Patch Blending Algorithm for Artefact Reduction in - - PowerPoint PPT Presentation

Sep 11, 2022 121 likes •543 views

A Fast Spatial Patch Blending Algorithm for Artefact Reduction in Pattern-based Image Inpainting Maxime Daisy, David Tschumperl e, Olivier L ezoray GREYC - UMR 6072 CNRS, ENSICAEN, University of Caen Image team SIGGRAPH Asia 21

slide-1
SLIDE 1

A Fast Spatial Patch Blending Algorithm for Artefact Reduction in Pattern-based Image Inpainting

Maxime Daisy, David Tschumperl´ e, Olivier L´ ezoray

GREYC - UMR 6072 CNRS, ENSICAEN, University of Caen Image team

SIGGRAPH Asia

21 November 2013

slide-2
SLIDE 2

Outlines

1 Inpainting : Context and State of the Art 2 Patch Blending for Inpainting 3 Results and Comparisons

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 2 / 41

slide-3
SLIDE 3

Outlines

1 Inpainting : Context and State of the Art 2 Patch Blending for Inpainting 3 Results and Comparisons

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 3 / 41

slide-4
SLIDE 4

Image Inpainting : Overview

Original

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 4 / 41

slide-5
SLIDE 5

Image Inpainting : Overview

Masked image

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 5 / 41

slide-6
SLIDE 6

Image Inpainting : Overview

Inpainted image

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 6 / 41

slide-7
SLIDE 7

Image Inpainting : a little history

  • Masnou and Morel : Level lines based disocclusion, ICIP 1998
  • Efros and Leung : Texture synthesis by non-parametric

sampling, ICCV 1999

  • Bertalm´

ıo et al. : Image Inpainting, SIGGRAPH 2000

  • Criminisi et al. : Object Removal by Exemplar-based

Inpainting, CVPR 2003

  • Wexler et al. : Space-time video completion, CVPR 2004
  • Sun et al. : Image Inpainting by Patch Propagation Using

Patch Sparsity IEEE 2010

  • Darabi et al. : Image Melding : Combining Inconsistent

Images using Patch-based Synthesis, SIGGRAPH 2012

  • . . .
  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 7 / 41

slide-8
SLIDE 8

Criminisi et al. : overview

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 8 / 41

slide-9
SLIDE 9

Good texture reconstruction

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 9 / 41

slide-10
SLIDE 10

Good texture reconstruction

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 10 / 41

slide-11
SLIDE 11

Local geometry inconsistencies

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 11 / 41

slide-12
SLIDE 12

Since Criminisi et al. 2003 inpainting algorithm

Content Aware Filling 2010

  • Wexler et al. 2007 : Space-time completion of videos
  • Barnes et al. 2009 : PatchMatch

⇒ blurry textures Le Meur et al. 2011

  • local geometry aware : structure tensors analysis
  • K-nearest neighbours averaging

⇒ blurry textures, geometry inconsitencies

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 12 / 41

slide-13
SLIDE 13

Since Criminisi et al. 2003 inpainting algorithm

(a) Content-aware filling (b) Le Meur et al. 2011

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 13 / 41

slide-14
SLIDE 14

The question

How to keep both good textures and geometry consistency ?

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 14 / 41

slide-15
SLIDE 15

Outlines

1 Inpainting : Context and State of the Art 2 Patch Blending for Inpainting 3 Results and Comparisons

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 15 / 41

slide-16
SLIDE 16

Spatial patch blending - Extreme case

Masked image.

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 16 / 41

slide-17
SLIDE 17

Spatial patch blending - Extreme case

Inpainted image : Criminis et al. 2003.

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 17 / 41

slide-18
SLIDE 18

Spatial patch blending - Extreme case

Diffusion PDE inpainting result.

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 18 / 41

slide-19
SLIDE 19

Spatial patch blending - Extreme case

Content-aware filling algorithm.

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 19 / 41

slide-20
SLIDE 20

Spatial patch blending - Extreme case

Effect of spatial patch blending.

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 20 / 41

slide-21
SLIDE 21

σ(p) : an adaptative blending amplitude

(c) Break field R(p) (d) Blending amplitude field σ(p)

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 21 / 41

slide-22
SLIDE 22

Spatial Patch Blending : explanation

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 22 / 41

slide-23
SLIDE 23

Mathematical expression

  • Usage of reconstruction points and patch locations, saved

during inpainting process Ji(p) = ∑

ψq∈Ψp

w(q,p) ψi

q(p −q)

ε + ∑

ψq∈Ψp

w(q,p)

  • Ψp : set of patches used for the reconstruction around p
  • ψi

q(p −q) : pixel value at ((N/2,N/2)T +(p −q)) in ψi q

  • σ(p) : blending amplitude at p
  • w(q,p) gaussian weight of variance σ(p)2
  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 23 / 41

slide-24
SLIDE 24

Fast algorithm - Blend scales

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 24 / 41

slide-25
SLIDE 25

Outlines

1 Inpainting : Context and State of the Art 2 Patch Blending for Inpainting 3 Results and Comparisons

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 25 / 41

slide-26
SLIDE 26

Time Comparison

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 26 / 41

slide-27
SLIDE 27

Result - Nature

Original Image

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 27 / 41

slide-28
SLIDE 28

Result - Nature

Content-aware Filling

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 28 / 41

slide-29
SLIDE 29

Result - Nature

Criminisi + our spatial patch blending

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 29 / 41

slide-30
SLIDE 30

Result - Fake Owl

Original Image

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 30 / 41

slide-31
SLIDE 31

Result - Fake Owl

Criminisi et al. 2004

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 31 / 41

slide-32
SLIDE 32

Result - Fake Owl

Content-aware Filling

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 32 / 41

slide-33
SLIDE 33

Result - Fake Owl

Criminisi + our spatial patch blending

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 33 / 41

slide-34
SLIDE 34

Result - Savanna

Original Image

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 34 / 41

slide-35
SLIDE 35

Result - Savanna

Criminisi et al. 2005

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 35 / 41

slide-36
SLIDE 36

Result - Savanna

Content-aware Filling

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 36 / 41

slide-37
SLIDE 37

Result - Savanna

Criminisi + our spatial patch blending

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 37 / 41

slide-38
SLIDE 38

GIMP Software Integration : G’MIC plugin

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 38 / 41

slide-39
SLIDE 39

GIMP Software Integration : G’MIC plugin

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 39 / 41

slide-40
SLIDE 40

GIMP Software Integration : G’MIC plugin

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 40 / 41

slide-41
SLIDE 41

GIMP Software Integration : G’MIC plugin

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 41 / 41

slide-42
SLIDE 42

GIMP Software Integration : G’MIC plugin

  • M. Daisy (GREYC)

SIGGRAPH Asia 2013 21 November 2013 42 / 41