Texture CS 419 Slides by Ali Farhadi What is a Texture? Texture - - PowerPoint PPT Presentation

▶
texture
SMART_READER_LITE
LIVE PREVIEW

Texture CS 419 Slides by Ali Farhadi What is a Texture? Texture - - PowerPoint PPT Presentation

Dec 27, 2023 297 likes •1.1k views

Texture CS 419 Slides by Ali Farhadi What is a Texture? Texture Spectrum Steven Li, James Hays, Chenyu Wu, Vivek Kwatra, and Yanxi Liu, CVPR 06 Texture scandals!! Two crucial algorithmic points Nearest neighbors again and again

slide-1
SLIDE 1

Texture

CS 419 Slides by Ali Farhadi

slide-2
SLIDE 2

What is a Texture?

slide-3
SLIDE 3

Texture Spectrum

Steven Li, James Hays, Chenyu Wu, Vivek Kwatra, and Yanxi Liu, CVPR 06

slide-4
SLIDE 4

Texture scandals!!

slide-5
SLIDE 5
slide-6
SLIDE 6

Two crucial algorithmic points

  • Nearest neighbors
  • again and again and again
  • Dynamic programming
  • likely new; we’ll use this again, too
slide-7
SLIDE 7

Texture Synthesis

Efros & Leung ICCV99

slide-8
SLIDE 8

How to paint this pixel?

? Efros & Leung ICCV99 Input texture p

slide-9
SLIDE 9

Ask Neighbors

  • What is the conditional probability distribution of p,

given it’s neighbors?

p Efros & Leung ICCV99 p

slide-10
SLIDE 10
  • Don’t bother to model the distribution
  • It’s already there, in the image

Input image Efros & Leung ICCV99

slide-11
SLIDE 11

Efros & Leung Algorithm p Synthesizing a pixel non-parametric sampling Input image Efros & Leung ICCV99

slide-12
SLIDE 12

Concerns

  • Distance metric
  • Neighborhood size
  • Order to paint
slide-13
SLIDE 13

Distance metric

  • Normalized sum of squared distances
  • Not all the neighbors worth the same
  • Gaussian mask
  • Preserve the local structure
  • Pick among reasonably similar neighborhoods
slide-14
SLIDE 14

input Efros & Leung ICCV99

Neighborhood size

slide-15
SLIDE 15

Varying Window Size

Increasing window size Efros & Leung ICCV99

slide-16
SLIDE 16

The Order matters

slide-17
SLIDE 17

Efros & Leung ICCV99

Some Results

slide-18
SLIDE 18

Efros & Leung ICCV99

More Results

slide-19
SLIDE 19

More Results

french canvas rafia weave Efros & Leung ICCV99

slide-20
SLIDE 20

More Results

wood granite Efros & Leung ICCV99

slide-21
SLIDE 21

More Results

white bread brick wall Efros & Leung ICCV99

slide-22
SLIDE 22

Growing Regions Hole Filling

Efros & Leung ICCV99

slide-23
SLIDE 23

Hole Filling

Efros & Leung ICCV99

slide-24
SLIDE 24

Extrapolation

Efros & Leung ICCV99

slide-25
SLIDE 25

Failure Cases

Growing garbage Verbatim copying Efros & Leung ICCV99

slide-26
SLIDE 26

Pros and Cons

  • Very simple
  • Easy to implement
  • Promising results
  • Very sloooooooowwwwwww
  • Idea:
  • Patches instead of pixels
slide-27
SLIDE 27

Patch based

  • Observation
  • neighbouring pixels are highly correlated
  • Idea:
  • unit of synthesis = block

Input image non-parametric sampling B Synthesizing a block Efros & Freeman SIGGRAPH01

slide-28
SLIDE 28

Input texture B1 B2 Random placement

  • f blocks

block B1 B2 Neighboring blocks constrained by overlap B1 B2 Minimal error boundary cut Efros & Freeman SIGGRAPH01

slide-29
SLIDE 29
  • min. error boundary

Minimal error boundary

  • verlapping blocks

vertical boundary _ = 2

  • verlap error

Efros & Freeman SIGGRAPH01

slide-30
SLIDE 30

Dynamic Programming

S T

slide-31
SLIDE 31

Dynamic Programming

S T

slide-32
SLIDE 32

B1 B2 Minimal error boundary cut B1 B2 Random placement

  • f blocks

B1 B2 Neighboring blocks constrained by overlap Efros & Freeman SIGGRAPH01

slide-33
SLIDE 33

Efros & Freeman SIGGRAPH01

More Results

slide-34
SLIDE 34

Efros & Freeman SIGGRAPH01

More Results

slide-35
SLIDE 35

Efros & Freeman SIGGRAPH01

slide-36
SLIDE 36

Efros & Freeman SIGGRAPH01

slide-37
SLIDE 37

Efros & Freeman SIGGRAPH01

slide-38
SLIDE 38

Efros & Freeman SIGGRAPH01

slide-39
SLIDE 39

Efros & Freeman SIGGRAPH01

Failures

slide-40
SLIDE 40

+ =

Texture Transfer

  • Take the texture from on object and paint it on another
  • bject

Decomposing shape and texture Very challenging Walk around Add some constraint to the search Efros & Freeman SIGGRAPH01

slide-41
SLIDE 41

Source Texture Source Map Destination Map Destination

slide-42
SLIDE 42
slide-43
SLIDE 43

= +

Texture Transfer

Efros & Freeman SIGGRAPH01

slide-44
SLIDE 44

+ = Efros & Freeman SIGGRAPH01

slide-45
SLIDE 45

Efros & Freeman SIGGRAPH01

slide-46
SLIDE 46

+ = + = parmesan rice Efros & Freeman SIGGRAPH01

slide-47
SLIDE 47

Image Analogies

? Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-48
SLIDE 48

Image Analogies

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-49
SLIDE 49

Image Analogies

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-50
SLIDE 50

Image Analogies

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-51
SLIDE 51

Training

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-52
SLIDE 52

: B B’ :: Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-53
SLIDE 53

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-54
SLIDE 54

: B B’ :: Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-55
SLIDE 55

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-56
SLIDE 56

Learn to Blur

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-57
SLIDE 57

Texture by Numbers

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-58
SLIDE 58

Colorization

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-59
SLIDE 59

Super-resolution

A A’ Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-60
SLIDE 60

Super-resolution (result!)

B B’ Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-61
SLIDE 61

Training images

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-62
SLIDE 62

Hertzman, Jacobs, Oliver, Curless, and Salesin, SIGGRAPH01

slide-63
SLIDE 63

Inpainting

Criminisi et.al. CVPR03

slide-64
SLIDE 64

Order of inpainting matters

Criminisi et al, 04

slide-65
SLIDE 65

Choosing the order

Criminisi et al 03

slide-66
SLIDE 66

Constraining the match region

  • We don’t have to look for matches in the whole image
  • idea: allow user to “paint” good sources of matches on top of the image
slide-67
SLIDE 67

Nearest Neighbor search

The core of most of the patch based methods Very slow Smarter neighborhood search Barnes et.al. SIGGRAPH09

slide-68
SLIDE 68

Inpainting

Barnes et.al. SIGGRAPH09

slide-69
SLIDE 69

Applications

Barnes et.al. SIGGRAPH09

slide-70
SLIDE 70

Retargeting

  • Make an image bigger or smaller in one direction
  • eg change aspect ratio
  • Traditional
  • cut off pixels
  • difficulty: lousy results
  • Strategy
  • cut out a curve of pixels that “doesn’t matter much”
  • low energy at pixels
  • many energy functions, eg
slide-71
SLIDE 71

Finding a seam=DP

Avidan, Shamir, SIGGRAPH07

slide-72
SLIDE 72
  • Different energies give

different results

  • e1 = abs gradient (as above)
  • ehog = (look for gradients in

patch)

  • eentropy = (entropy of patch)
  • eseg = (segment image, e1 in

segments, 0 on boundaries)

slide-73
SLIDE 73

Retargeting

Seam removal Scaling Cropping Avidan, Shamir, SIGGRAPH07

slide-74
SLIDE 74

Retargeting

Avidan, Shamir, SIGGRAPH07

slide-75
SLIDE 75

Avidan, Shamir, SIGGRAPH07

slide-76
SLIDE 76

Can use constraints in retargeting

Barnes et.al. SIGGRAPH09

slide-77
SLIDE 77

Constrained retargeting

Barnes et.al. SIGGRAPH09

slide-78
SLIDE 78

Local scale editing

Barnes et.al. SIGGRAPH09

slide-79
SLIDE 79

reshuffling

Barnes et.al. SIGGRAPH09

slide-80
SLIDE 80

Barnes et.al. SIGGRAPH09