Multi-perspective Panoramas Slides from a talk by Lihi Zelnik-Manor - - PowerPoint PPT Presentation

▶
multi perspective panoramas
SMART_READER_LITE
LIVE PREVIEW

Multi-perspective Panoramas Slides from a talk by Lihi Zelnik-Manor - - PowerPoint PPT Presentation

Feb 21, 2024 132 likes •893 views

Multi-perspective Panoramas Slides from a talk by Lihi Zelnik-Manor at ICCV07 3DRR workshop Pictures capture memories Panoramas Registration: Brown & Lowe, ICCV05 Blending: Burt & Adelson, Trans. Graphics,1983

slide-1
SLIDE 1

Multi-perspective Panoramas

Slides from a talk by Lihi Zelnik-Manor at ICCV’07 3DRR workshop

slide-2
SLIDE 2

Pictures capture memories

slide-3
SLIDE 3

Panoramas

Registration: Brown & Lowe, ICCV’05 Blending: Burt & Adelson, Trans. Graphics,1983 Visualization: Kopf et al., SIGGRAPH, 2007

slide-4
SLIDE 4

Bad panorama?

Output of Brown & Lowe software

slide-5
SLIDE 5

No geometrically consistent solution

slide-6
SLIDE 6

Scientists solution to panoramas: Single center of projection

Registration: Brown & Lowe, ICCV’05 Blending: Burt & Adelson, Trans. Graphics,1983 Visualization: Kopf et al., SIGGRAPH, 2007

No 3D!!!

slide-7
SLIDE 7

From sphere to plane

Distortions are unavoidable

slide-8
SLIDE 8

Distorted panoramas

Output of Brown & Lowe software

Actual appearance

slide-9
SLIDE 9

Objectives

  • 1. Better looking panoramas
  • 2. Let the camera move:
  • Any view
  • Natural photographing
slide-10
SLIDE 10

Stand on the shoulders of giants

Cartographers Artists

slide-11
SLIDE 11

Cartographic projections

slide-12
SLIDE 12

Common panorama projections

θ φ

Cylindircal Perspective Stereographic

slide-13
SLIDE 13

Global Projections

Cylindircal Perspective Stereographic

slide-14
SLIDE 14

Learn from the artists

Multiple view points

De Chirico “Mystery and Melancholy of a Street”, 1914

perspective perspective Sharp discontinuity

slide-15
SLIDE 15

Two horizons!

slide-16
SLIDE 16

Renaissance painters solution

“School of Athens”, Raffaello Sanzio ~1510

Give a separate treatment to different parts of the scene!!

slide-17
SLIDE 17

Personalized projections

“School of Athens”, Raffaello Sanzio ~1510

Give a separate treatment to different parts of the scene!!

slide-18
SLIDE 18

Multiple planes of projection

Sharp discontinuities can often be well hidden

slide-19
SLIDE 19

Our multi-view result Single view

slide-20
SLIDE 20

Our multi-view result Single view

slide-21
SLIDE 21

Our multi-view result Single view

slide-22
SLIDE 22

Applying personalized projections

Foreground Input images Background panorama

slide-23
SLIDE 23

Single view Our multi-view result

slide-24
SLIDE 24

Single view Our multi-view result

slide-25
SLIDE 25

Objectives - revisited

  • 1. Better looking panoramas
  • 2. Let the camera move:
  • Any view
  • Natural photographing

Multiple views can live together

slide-26
SLIDE 26

Multi-view compositions

David Hockney, Place Furstenberg, (1985)

slide-27
SLIDE 27

Melissa Slemin, Place Furstenberg, 2003

Why multi-view?

Multiple viewpoints Single viewpoint

David Hockney, Place Furstenberg, 1985

slide-28
SLIDE 28

Multi-view panoramas

Single view Multiview

Requires video input

Zomet et al. (PAMI’03)

slide-29
SLIDE 29

Long Imaging

Agarwala et al. (SIGGRAPH 2006)

slide-30
SLIDE 30

Smooth Multi-View

Google maps

slide-31
SLIDE 31

What’s wrong in the picture?

Google maps

slide-32
SLIDE 32

Non-smooth

Google maps

slide-33
SLIDE 33

The Chair

David Hockney (1985)

slide-34
SLIDE 34

Joiners are popular

4,985 photos matching joiners. 4,007 photos matching Hockney. 41 groups about Hockney Thousands of members Flickr statistics (Aug’07):

slide-35
SLIDE 35

Main goals: Automate joiners Generalize panoramas to general image collections

slide-36
SLIDE 36

Objectives

  • For Artists:

Reduce manual labor Manual: ~40min. Fully automatic

slide-37
SLIDE 37

Objectives

  • For Artists:

Reduce manual labor

  • For non-artists:

Generate pleasing-to-the-eye joiners

slide-38
SLIDE 38

Objectives

  • For Artists:

Reduce manual labor

  • For non-artists:

Generate pleasing-to-the-eye joiners

  • For data exploration:

Organize images spatially

slide-39
SLIDE 39

What’s going on here?

slide-40
SLIDE 40

A cacti garden

slide-41
SLIDE 41

Principles

slide-42
SLIDE 42

Principles

  • Convey topology

Correct Incorrect

slide-43
SLIDE 43

Principles

  • Convey topology
  • A 2D layering of images

Blending: blurry Graph-cut: cuts hood Desired joiner

slide-44
SLIDE 44

Principles

  • Convey topology
  • A 2D layering of images
  • Don’t distort images

rotate scale translate

slide-45
SLIDE 45

Principles

  • Convey topology
  • A 2D layering of images
  • Don’t distort images
  • Minimize inconsistencies

Good Bad

slide-46
SLIDE 46

Algorithm

slide-47
SLIDE 47

Step 1: Feature matching

Brown & Lowe, ICCV’03

slide-48
SLIDE 48

Step 2: Align

Large inconsistencies Brown & Lowe, ICCV’03

slide-49
SLIDE 49

Step 3: Order

Reduced inconsistencies

slide-50
SLIDE 50

Ordering images

Try all orders: only for small datasets

slide-51
SLIDE 51

Ordering images

Try all orders: only for small datasets complexity: (m+n) m = # images n = # overlaps = # acyclic orders

slide-52
SLIDE 52

Ordering images

Observations: – Typically each image overlaps with only a few others – Many decisions can be taken locally

slide-53
SLIDE 53

Ordering images

Approximate solution: – Solve for each image independently – Iterate over all images

slide-54
SLIDE 54

Can we do better?

slide-55
SLIDE 55

Step 4: Improve alignment

slide-56
SLIDE 56

Iterate Align-Order-Importance

slide-57
SLIDE 57

Iterative refinement

Initial Final

slide-58
SLIDE 58

Iterative refinement

Initial Final

slide-59
SLIDE 59

Iterative refinement

Initial Final

slide-60
SLIDE 60

What is this?

slide-61
SLIDE 61

That’s me reading

slide-62
SLIDE 62

Anza-Borrego

slide-63
SLIDE 63

Tractor

slide-64
SLIDE 64

Paolo Uccello, 1436

Art reproduction

slide-65
SLIDE 65

Paolo Uccello, 1436 Zelnik & Perona, 2006

Art reproduction

slide-66
SLIDE 66

Single view-point Zelnik & Perona, 2006

Art reproduction

slide-67
SLIDE 67

Manual by Photographer

slide-68
SLIDE 68

Our automatic result

slide-69
SLIDE 69

Failure?

slide-70
SLIDE 70

GUI

slide-71
SLIDE 71

The Impossible Bridge

slide-72
SLIDE 72

Homage to David Hockney

slide-73
SLIDE 73
  • Incorrect geometries are possible and fun!
  • Geometry is not enough, we need scene

analysis

  • A highly related work:

"Scene Collages and Flexible Camera Arrays,”

  • Y. Nomura, L. Zhang and S.K. Nayar,

Eurographics Symposium on Rendering, Jun, 2007.

Take home

slide-74
SLIDE 74

Thank You

slide-75
SLIDE 75

15-463 Class Project from 2007

http://www.cs.cmu.edu/afs/andrew/scs/cs/1 5-463/f07/proj_final/www/echuangs/