Sisley the Abstract Painter Mingtian Zhao Song-Chun Zhu University - - PowerPoint PPT Presentation

sisley the abstract painter
SMART_READER_LITE
LIVE PREVIEW

Sisley the Abstract Painter Mingtian Zhao Song-Chun Zhu University - - PowerPoint PPT Presentation

NPAR 2010 Sisley the Abstract Painter Mingtian Zhao Song-Chun Zhu University of California, Los Angeles & Lotus Hill Institute Motivation Wheatstack (Thaw, Sunset) Claude Monet 189091 I considered that the painter had no right to


slide-1
SLIDE 1

NPAR 2010

Sisley the Abstract Painter

Mingtian Zhao Song-Chun Zhu University of California, Los Angeles & Lotus Hill Institute

slide-2
SLIDE 2

Motivation

Wheatstack (Thaw, Sunset) Claude Monet 1890–91 “I considered that the painter had no right to paint indistinctly . . . and I noticed with surprise and confusion that the picture not only gripped me, but impressed itself ineradicably on my memory.” — Wassily Kandinsky

slide-3
SLIDE 3

Perceptual Ambiguity

  • Indistinction, Confusion: Perceptual Ambiguity
  • The Mechanism of Abstract Arts [Berlyne 1971]

Perceptual Ambiguity ⇒ Mental Efforts ⇒ Arousal Changes ⇒ Aesthetic Pleasures

  • Where does perceptual ambiguity come from?

what we see = arg max P(interpretation|image)

slide-4
SLIDE 4

More Abstract Arts

  • No. 5, 1948

Jackson Pollock Violin and Guitar Pablo Picasso

slide-5
SLIDE 5

More Abstract Arts

Le Mont Sainte-Victoire Paul C´ ezanne The Red Vineyard Vincent van Gogh

slide-6
SLIDE 6

More Abstract Arts

  • They all preserve certain features and free others.
  • Features are marginal statistics.
  • Projection Pursuit [Friedman & Tukey 1974]
  • Semantic Fidelity vs. Uncertainty & Ambiguity
  • Paths of Perception
slide-7
SLIDE 7

Our Approach

  • Interactive Image Parsing [Tu et al. 2005]

⋄ Interactive Segmentation ⋄ Hierarchical Organization ⋄ Category Labeling (optional but recommended)

  • Customization and Rendering

⋄ Customized Perceptual Ambiguity Levels ⋄ Stochastic Operations on Color/Shape/Texture

  • Computation and Control

⋄ Kernel Density Estimation ⋄ Belief Propagation ⋄ Servomechanism

slide-8
SLIDE 8

Parse Tree

seascape sailboat sea buildings trees sky sail hull

slide-9
SLIDE 9

Numerical Measure of Perceptual Ambiguity

  • Assume parse tree structure is obvious, and
  • Perceptual ambiguity is only with categories

L = (ℓ1, ℓ2, · · · , ℓK)

  • Uncertainty in p(L): Information/Shannon Entropy

H(L)|I =

  • L

−p(L|I) log p(L|I)

slide-10
SLIDE 10

Stochastic Operations

  • Color

⋄ hue shift ∆h ∼ GT(0, σ2, −3σ, 3σ) σ ∝ H, σmax = 15◦

  • Shape

⋄ boundary pixel shift (∆x, ∆y ∼ GT) ⋄ image warping

  • Texture

⋄ Painterly Rendering (adapted from [Zeng et al. 2009])

slide-11
SLIDE 11

Computation & Control

  • Parse Tree as Markov random field (MRF)

⋄ Hammersley-Clifford p(L|I) = 1 Z

  • i∈V

φi(ℓi)

  • i,j∈E

ψij(ℓi, ℓj) ⋄ Unary Term: Local Evidence φi(ℓi) ← p(ℓi|Ii) ⋄ Binary Term: Compatibility ψij(ℓi, ℓj) ← f(ℓi, ℓj)

slide-12
SLIDE 12

Computation & Control

  • Local Evidence by Kernel Density Estimation

Distant: low-weight votes Close: high-weight votes

  • The LHI Image Database
slide-13
SLIDE 13

Computation & Control

  • Local Evidence by Kernel Density Estimation

p(ℓi|Ii) ∝

  • n

ϕ(Ii, Jn)1(ℓi = ℓn)

  • The Exponential Kernel

ϕ(Ii, Jn) = exp {−λs(Ii) − s(Jn)} s : Opponent-SIFT [van de Sande et al. 2010]

  • The use of manually labeled categories
  • Can we plug in p(ℓi|Ii) to get p(L|I) and H(L)|I?
slide-14
SLIDE 14

Computation & Control

  • Problem of computing H(L)|I:

The space of L is usually huge. |ΩL| = (#categories)#nodes

  • Approximation with p(ℓi|I) instead of p(ℓi|Ii)

H(L)|I ≈

  • i

wiH(ℓi)|I wi ∝ size(i) H(ℓi)|I =

  • L

−p(ℓi|I) log p(ℓi|I)

slide-15
SLIDE 15

Computation & Control

  • Belief Propagation on MRF [Yedidia et al. 2001]

Ii i bi(ℓi) j k mij mji mik mki p(ℓi|Ii)

slide-16
SLIDE 16

Computation & Control

  • Belief Propagation on MRF [Yedidia et al. 2001]

⋄ Update Messages mij(ℓj) =

  • ℓi

p(ℓi|Ii) f(ℓi, ℓj)

  • k∈∂i\j

mki(ℓi) ⋄ Update Beliefs bi(ℓi) ∝ p(ℓi|Ii)

  • j∈∂i

mji(ℓi) ⋄ p(ℓi|I) ← bi(ℓi) after convergence

slide-17
SLIDE 17

Computation & Control

  • Relative Abstract Level
  • H =
  • i wiH(ℓi)|I
  • i wi log
  • Ωℓi
  • ∈ [0, 1]
  • Servomechanism

⋄ If H(t)

  • ut <

H(t)

in

  • H(t+1)

I

= H(t)

in

2

  • H(t)
  • ut

⋄ If H(t)

  • ut >

H(t)

in

  • H(t+1)

in

= 1 −

  • 1 −

H(t)

in

2 1 − H(t)

  • ut
slide-18
SLIDE 18

Experimental Results

  • H ≈ 0
  • H ≈ 0.25
  • H ≈ 0.5
  • H ≈ 0.75
slide-19
SLIDE 19

Experimental Results

slide-20
SLIDE 20

Experimental Results

slide-21
SLIDE 21

Experimental Results

Different Paths of Perception

slide-22
SLIDE 22

Evaluations

  • Comparative Human Experiments

Photographs Original Paintings Synthesized Paintings Alley Flying Bird Buildings Butterfly

slide-23
SLIDE 23

Evaluations

  • Recognition Accuracy

Photographs Original Paintings Synthesized Paintings Horizontal axis: reported categories Vertical axis: true categories Darkness of grids: frequencies

slide-24
SLIDE 24

Evaluations

  • Response Speed

⋄ ANOVA F-test: p-value=2.955 × 10−8 ⋄ Tukey’s HSD

Group Pair ∆¯ t (ms) p-value Photographs vs. Original Paintings 2165 < 0.01 Photographs vs. Synthesized Paintings 1183 0.03 Original vs. Synthesized Paintings −982 0.11

slide-25
SLIDE 25

Summary & Future Studies

  • Abstract Arts Rendering by Perceptual Ambiguity

Computation and Control

  • Convergence control
  • Variable parse tree structures
  • Mixture modeling of abstract arts

More Results and Demo System: http://www.stat.ucla.edu/∼mtzhao/research/sisley/