REGISTRATION FOR MOBILE HDR PHOTOGRAPHY Orazio Gallo, 04/06/2016 - - PowerPoint PPT Presentation

▶
registration for mobile hdr
SMART_READER_LITE
LIVE PREVIEW

REGISTRATION FOR MOBILE HDR PHOTOGRAPHY Orazio Gallo, 04/06/2016 - - PowerPoint PPT Presentation

Dec 09, 2023 49 likes •760 views

April 4-7, 2016 | Silicon Valley LOCALLY NON-RIGID REGISTRATION FOR MOBILE HDR PHOTOGRAPHY Orazio Gallo, 04/06/2016 (work with Alejandro Troccoli, Jun Hu, Kari Pulli, and Jan Kautz) WHAT IS HIGH -DYNAMIC- RANGE? Kinda depends on whom you

slide-1
SLIDE 1

April 4-7, 2016 | Silicon Valley

Orazio Gallo, 04/06/2016

(work with Alejandro Troccoli, Jun Hu, Kari Pulli, and Jan Kautz)

LOCALLY NON-RIGID REGISTRATION FOR MOBILE HDR PHOTOGRAPHY

slide-2
SLIDE 2

2

Displays Videogames Photography Photometry … WHAT IS “HIGH-DYNAMIC-RANGE”?

Kinda depends on whom you ask…

slide-3
SLIDE 3

3

WHAT IS HDR?

http://www.flickr.com/photos/lprowler/5704117093/

slide-4
SLIDE 4

4

WHAT IS HDR?

slide-5
SLIDE 5

6

WHAT IS HDR?

slide-6
SLIDE 6

7

WHY DO WE NEED REGISTRATION?

slide-7
SLIDE 7

8

WHY DO WE NEED REGISTRATION?

slide-8
SLIDE 8

9

WHY DO WE NEED REGISTRATION?

slide-9
SLIDE 9

10

WHY DO WE NEED REGISTRATION?

slide-10
SLIDE 10

11

RELATED WORK

slide-11
SLIDE 11

12

FULLY NON-RIGID REGISTRATION

Hu, Gallo, Pulli, Sun, “HDR Deghosting: How to Deal with Saturation?” IEEE CVPR 2013.

slide-12
SLIDE 12

13

FULLY NON-RIGID REGISTRATION

Hu, Gallo, Pulli, Sun, “HDR Deghosting: How to Deal with Saturation?” IEEE CVPR 2013.

slide-13
SLIDE 13

14

FULLY NON-RIGID REGISTRATION

Hu, Gallo, Pulli, Sun, “HDR Deghosting: How to Deal with Saturation?” IEEE CVPR 2013.

slide-14
SLIDE 14

15

FULLY NON-RIGID REGISTRATION

Hu, Gallo, Pulli, Sun, “HDR Deghosting: How to Deal with Saturation?” IEEE CVPR 2013.

slide-15
SLIDE 15

16

FULLY NON-RIGID REGISTRATION

Hu, Gallo, Pulli, Sun, “HDR Deghosting: How to Deal with Saturation?” IEEE CVPR 2013.

slide-16
SLIDE 16

17

FULLY NON-RIGID REGISTRATION

Hu, Gallo, Pulli, Sun, “HDR Deghosting: How to Deal with Saturation?” IEEE CVPR 2013.

slide-17
SLIDE 17

19

Quality Speed Rigid registration

(Ward ‘03, Tomaszewska and Mantiuk ‘07, …)

Fully non-rigid registration

(Hu et al. ‘12, Sen et al. ‘12, Hu et al. ‘13)

Rejection methods

(Gallo ‘09, Zhang and Cham ‘12, Oh et al. ‘15 …)

Flow-based methods

(Zimmer et al. ‘11, Zhang and Cham ‘12,…)

Accelerated patch- based

(Bao et al. ‘14)

Minutes Seconds Milliseconds Little to no artifacts Parallax and motion artifacts Motion artifacts

  • r

reduced dynamic range

RELATED WORK

slide-18
SLIDE 18

25

WHAT’S THE CATCH?

SIFT+warp Original Ours

slide-19
SLIDE 19

28

A SPARSE-TO-DENSE APPROACH

Compute the flow at sparse locations, Propagate the flow in an edge-aware fashion, and Merge the images in an error-tolerant way.

slide-20
SLIDE 20

29

METHOD

slide-21
SLIDE 21

30

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-22
SLIDE 22

31

Reference

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

Source

slide-23
SLIDE 23

34

Reference Source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-24
SLIDE 24

35

Reference Source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-25
SLIDE 25

39

Reference

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-26
SLIDE 26

40

Source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-27
SLIDE 27

41

Reference

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-28
SLIDE 28

42

Source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-29
SLIDE 29

43

Reference

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-30
SLIDE 30

44

Source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-31
SLIDE 31

57

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

Reference’s Luma Sparse flow

slide-32
SLIDE 32

59

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion sparse samples latent signal

slide-33
SLIDE 33

60

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion filtered sparse samples

slide-34
SLIDE 34

61

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion normalization map sparse samples

slide-35
SLIDE 35

62

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion filtered normalization map filtered sparse samples reconstructed signal

slide-36
SLIDE 36

67

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion Luminance Pixel

slide-37
SLIDE 37

68

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion Luminance Pixel

slide-38
SLIDE 38

69

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion Luminance Pixel Gastal and Oliveira, “Domain transform for edge-aware image and video processing” (SIGGRAPH '11)

slide-39
SLIDE 39

70

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion u v 1 1

Reference Luma, L Sparse flow, f Normalization map N

slide-40
SLIDE 40

71

Reference

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-41
SLIDE 41

72

Source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-42
SLIDE 42

73

Reference

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-43
SLIDE 43

74

Warped source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-44
SLIDE 44

75

Warped source

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-45
SLIDE 45

76

Corners and matches Filter matches Sparse-to- dense warp Error-tolerant fusion

slide-46
SLIDE 46

77

PERFORMANCE

slide-47
SLIDE 47

78

Execution time for a pair of 5MP images

slide-48
SLIDE 48

79

Quality Speed

Rigid registration

(Ward ‘03, Tomaszewska and Mantiuk ‘07, …)

Rejection methods

(Gallo ‘09, Zhang and Cham ‘12, Oh et al. ‘15 …)

Flow-based methods

(Zimmer et al. ‘11, Zhang and Cham ‘12,…)

Related work

Minutes Seconds Milliseconds No artifacts Parallax and motion artifacts Motion artifacts

  • r

reduced dynamic range

Accelerated patch-based

(Bao et al. ‘14)

Fully non-rigid registration

(Hu et al. ‘12, Sen et al. ‘12, Hu et al. ‘13)

slide-49
SLIDE 49

80

Quality Speed

Related work

Minutes Seconds Milliseconds No artifacts Parallax and motion artifacts Motion artifacts

  • r

reduced dynamic range

Accelerated patch-based

(Bao et al. ‘14)

Fully non-rigid registration

(Hu et al. ‘12, Sen et al. ‘12, Hu et al. ‘13)

(VGA resolution)

slide-50
SLIDE 50

81

Quality Speed

Related work

Minutes Seconds Milliseconds No artifacts Parallax and motion artifacts Motion artifacts

  • r

reduced dynamic range

Accelerated patch-based

(Bao et al. ‘14)

Fully non-rigid registration

(Hu et al. ‘12, Sen et al. ‘12, Hu et al. ‘13)

(5MP resolution)

slide-51
SLIDE 51

82

What about visually? Ours Bao et al.’s

(2.5MP)

slide-52
SLIDE 52

83

TO SUM UP…

Contributions

A fast registration algorithm >11x faster than the fastest published method We propose to use a sparse-to-dense approach CUDA-based sparse-to-dense propagation CUDA-based robust image fusion

slide-53
SLIDE 53

85

MORE RESULTS

Reference

slide-54
SLIDE 54

86

MORE RESULTS

Source

slide-55
SLIDE 55

87

MORE RESULTS

Reference

slide-56
SLIDE 56

88

MORE RESULTS

Warped source

slide-57
SLIDE 57

89

MORE RESULTS

HDR

slide-58
SLIDE 58

90

MORE RESULTS

Naïve fusion Our result

slide-59
SLIDE 59

91

MORE RESULTS

Reference

slide-60
SLIDE 60

92

MORE RESULTS

Source

slide-61
SLIDE 61

93

MORE RESULTS

Reference

slide-62
SLIDE 62

94

MORE RESULTS

Warped source

slide-63
SLIDE 63

95

MORE RESULTS

HDR

slide-64
SLIDE 64

96

MORE RESULTS

Naïve fusion Our result

slide-65
SLIDE 65

97

MORE RESULTS

Reference

slide-66
SLIDE 66

98

MORE RESULTS

Source

slide-67
SLIDE 67

99

MORE RESULTS

Reference

slide-68
SLIDE 68

100

MORE RESULTS

Warped source

slide-69
SLIDE 69

101

MORE RESULTS

HDR

slide-70
SLIDE 70

102

MORE RESULTS

Naïve fusion Our result

slide-71
SLIDE 71

April 4-7, 2016 | Silicon Valley

THAT’S ALL.

  • K. Pulli
  • A. Troccoli
  • J. Hu
  • J. Kautz
  • O. Gallo