3D Interest Maps from Simultaneous Video Recordings Axel Carlier - - PowerPoint PPT Presentation

▶
3d interest maps from simultaneous video recordings
SMART_READER_LITE
LIVE PREVIEW

3D Interest Maps from Simultaneous Video Recordings Axel Carlier - - PowerPoint PPT Presentation

Jul 09, 2023 313 likes •600 views

3D Interest Maps from Simultaneous Video Recordings Axel Carlier Lilian Calvet Universit de Toulouse Simula Research Laboratory Duong T. D. Nguyen and Pierre Gurdjos and Wei Tsang Ooi Vincent Charvillat National University of Singapore

slide-1
SLIDE 1

1

3D Interest Maps from Simultaneous Video Recordings

Axel Carlier

Université de Toulouse

Lilian Calvet

Simula Research Laboratory

Duong T. D. Nguyen and Wei Tsang Ooi

National University of Singapore

Pierre Gurdjos and Vincent Charvillat

Université de Toulouse

slide-2
SLIDE 2

2

Regions Of Interest

  • ROI: a region of a multimedia content that contains

semantic information that a user or a group of users may find interesting.

  • Highly subjective, dependent on

– Users – Context – Etc.

→ Difficult to predict automatically

slide-3
SLIDE 3

3

Related Work

  • Saliency detection
  • Usage-based approaches

Xie et al, CHI'05 Goferman et al, PAMI'12

slide-4
SLIDE 4

4

  • What people zoom into is interesting

[Carlier et al] Crowdsourced Automatic Zoom and Scroll for Video Retargeting, MM 2010

slide-5
SLIDE 5

5

Zoomable Video

slide-6
SLIDE 6

6

Viewports

slide-7
SLIDE 7

7

User interest maps

slide-8
SLIDE 8

8

From 2D to 3D

slide-9
SLIDE 9

9

Idea

  • What people choose to film is interesting
slide-10
SLIDE 10

10

The setup

  • Assumptions: calibration and synchronization

to a certain extent

slide-11
SLIDE 11

11

2D Regions Of Interest

With motion Without motion

slide-12
SLIDE 12

12

Back-Projection to 3D

slide-13
SLIDE 13

13

3D Interest Maps

  • Measure of the interest of a voxel

Set of 2D ROIs on all images Viewing cone of the ROI

slide-14
SLIDE 14

14

3D Interest Maps

  • Probabilistic form of the interest

A 3D interest map is the limit form of the 3D histogram with voxels as bins with respect to this measure.

slide-15
SLIDE 15

15

3D Interest Maps

  • We model our 3D interest map with a

Gaussian Mixture Model: How to estimate the GMM parameters?

slide-16
SLIDE 16

16

GMM estimation

Mean Shift Clustering 1 Cluster = 1 Gaussian

slide-17
SLIDE 17

17

Results

Coarse results, because of the lack of photometric consistency

slide-18
SLIDE 18

18

Photometric Consistency

High geometric consistency Low photometric consistency

[Furukawa et al] Accurate, dense, and robust multiview stereopsis, PAMI 2010 (a.k.a. PMVS software)

slide-19
SLIDE 19

19

New results

slide-20
SLIDE 20

20

Comparison with Saliency

slide-21
SLIDE 21

21

An application of 3D Interest Maps

slide-22
SLIDE 22

22

Mashup Video

[Saini et al] Movimash: online mobile video mashup, MM 2012

slide-23
SLIDE 23

23

Automatic Video Edition

3D transition

slide-24
SLIDE 24

24

slide-25
SLIDE 25

25

Result

slide-26
SLIDE 26

26

Evaluation

2D-VC: JikuDirector 2.0 (demo) 3D-VC: this paper

slide-27
SLIDE 27

27

Conclusion

  • Formal definition of 3D interest maps
  • A common space for representing interest in

many simultaneously recorded videos

– Thanks to our strong assumptions, this common

space is the 3D space

  • Many applications, including video mashup