TRECVID-2005: Shot Boundary Detection Task Overview Alan Smeaton - - PowerPoint PPT Presentation

▶
trecvid 2005 shot boundary detection task overview
SMART_READER_LITE
LIVE PREVIEW

TRECVID-2005: Shot Boundary Detection Task Overview Alan Smeaton - - PowerPoint PPT Presentation

Aug 13, 2023 262 likes •1.39k views

TRECVID-2005: Shot Boundary Detection Task Overview Alan Smeaton Dublin City University & Paul Over NIST SB Task Definition o Shot boundary detection is a fundamental task in any kind of video content manipulation o Task provides a

slide-1
SLIDE 1

TRECVID-2005: Shot Boundary Detection Task Overview

Alan Smeaton Dublin City University & Paul Over NIST

slide-2
SLIDE 2

TRECVID 2005 2

SB Task Definition

  • Shot boundary detection is a fundamental task in

any kind of video content manipulation

  • Task provides a good entry for groups who wish

to “break into” video retrieval and TRECVID gradually

  • Task is to identify the shot boundaries with their

location and type (cut or gradual) in the given video clip(s)

slide-3
SLIDE 3

TRECVID 2005 3

SB Task Details

  • Groups may submit up to 10 runs
  • Comparison to human-annotated reference

(thanks to Jonathan Lasko, again)

  • Groups were asked to provide some standard

information on the processing complexity of each run:

n Total runtime in seconds

  • Total decode time in seconds
  • Total segmentation time in seconds

n Processor description

slide-4
SLIDE 4

TRECVID 2005 4

Shot boundary task: Participating groups

City University of Hong Kong China SB LL -- -- CLIPS-IMAG, LSR-IMAG, Laboratoire LIS France SB –- HL -- Florida International University USA SB -- -- -- Fudan University China SB LL HL SE FX Palo Alto Laboratory USA SB –- HL SE Hong Kong Polytechnic University China SB -- -- -- IBM USA SB –- HL SE Imperial College London UK SB –- HL SE Indian Institute of Technology (IIT) India SB -- -- -- KDDI R\&D Laboratories, Inc. Japan SB LL -- -- LaBRI France SB LL -- -- Motorola Multimedia Research Laboratory USA SB -- -- -- National ICT Australia Australia SB LL HL -- RMIT University Australia SB -- -- -- Technical University of Delft Netherlands SB -- -- -- Tsinghua University China SB LL HL SE University of Central Florida / University of Modena USA,Italy SB LL HL SE University of Iowa USA SB LL -- SE University of Marburg Germany SB LL -- -- University Rey Juan Carlos Spain SB -- -- -- University of Sao Paulo (USP) Brazil SB -- -- --

slide-5
SLIDE 5

TRECVID 2005 5

Shot boundary data

q 12 representative videos (8 news, 4 NASA) q Total frames: 744,604

  • Total transitions: 4,535
  • 0.609 transitions/100frames (down from 0.777 in

2004)

  • Transition types:

n 2,759 (60.8%) Cuts (2004: 57.7%) n 1,382 (30.5%) Dissolves (2004:31.7%) n 81 (1.8%) Fade-out/-in (2004: 4.8%) n 313 (6.9%) other (2004: 5.7%)

slide-6
SLIDE 6

TRECVID 2005 6

Cuts

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-7
SLIDE 7

TRECVID 2005 7

Cuts (zoomed)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-8
SLIDE 8

TRECVID 2005 8

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-9
SLIDE 9

TRECVID 2005 9

2004: Cuts (zoomed again)

  • !"

#$$ % & ' ( ) !* )+,- (%! (./ ( $ (0

slide-10
SLIDE 10

TRECVID 2005 10 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-11
SLIDE 11

TRECVID 2005 11

Gradual transitions

  • !

" #

  • $
  • %&
slide-12
SLIDE 12

TRECVID 2005 12

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-13
SLIDE 13

TRECVID 2005 13

2004: Gradual transitions (zoomed)

  • !"

#$$ % & ' ( ) !* )+,- (%! (./ ( $ (0 )3

slide-14
SLIDE 14

TRECVID 2005 14

Evaluation Measures

Precision = Recall = Frame Precision = Frame Recall =

# Transitions Correctly Reported # Transitions Reported # Transitions Correctly Reported # Transitions in Reference # Frames Correctly Reported in Detected Transitions # Frames reported in Detected Transitions # Frames Correctly Reported in Detected Transitions # Frames in Reference Data for Detected Transitions

slide-15
SLIDE 15

TRECVID 2005 15

Gradual transitions (Frame-P & R)

  • !

" #

  • $
  • %&
slide-16
SLIDE 16

TRECVID 2005 16

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-17
SLIDE 17

TRECVID 2005 17

2004: Gradual Transitions (Frame-P&R)

  • !4

!4

  • !"

#$$ % & ' ( ) !* )+,- (%! (./ ( $ (0 )3

slide-18
SLIDE 18

TRECVID 2005 18

Results for News versus NASA videos – distribution

  • f per-file recall and precision by source type

Cuts Gradual

NASA News NASA News NASA News NASA News Recall Precision Recall Precision

slide-19
SLIDE 19

TRECVID 2005 19

Approaches

A roller-coaster through 21 groups’ submitted runs;

slide-20
SLIDE 20

TRECVID 2005 20

  • Approach

n Spatio-temporal (SD) slides are time vs. space representations

  • f video - shot transition types (cuts, dissolves) appear in SDs

with certain characteristics; Gabor features for motion texture and SVM for binary classification;

  • Features

n Expends previous (ACM MM) approach by including flash detection and extra visual features to discriminate GTs

  • Performance

n Because of image processing and SVM it is expensive;

  • Results
  • 1. City University of Hong Kong
slide-21
SLIDE 21

TRECVID 2005 21

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-22
SLIDE 22

TRECVID 2005 22

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-23
SLIDE 23

TRECVID 2005 23 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-24
SLIDE 24

TRECVID 2005 24

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-25
SLIDE 25

TRECVID 2005 25

  • Approach

n Appears to be a re-run of 2004 system, which was a re-run of 2003 (thanks for doing this) - emphasis was on features.

  • Features

n Detect cuts by image comparisons after motion compensation and GTs by comparing norms of first and second temporal derivatives of the images;

  • Performance

n About real-time, good on GTs;

  • Results
  • 2. CLIPS-IMAG, LSR-IMAG, Laboratoire LIS
slide-26
SLIDE 26

TRECVID 2005 26

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-27
SLIDE 27

TRECVID 2005 27

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-28
SLIDE 28

TRECVID 2005 28 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-29
SLIDE 29

TRECVID 2005 29

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-30
SLIDE 30

TRECVID 2005 30

  • Approach

n Didn’t submit a paper so we don’t know !

  • 3. Florida International University
slide-31
SLIDE 31

TRECVID 2005 31

Cuts (zoomed)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-32
SLIDE 32

TRECVID 2005 32 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-33
SLIDE 33

TRECVID 2005 33

  • Approach

n Frame-frame similarities, vary thresholds, use SVM classifier; n Explore HSV vs. LAB colour spaces;

  • Features

n Fudan definition of a short GT is a cut, differs from TRECVid evaluation, hence results depressed;

  • Performance

n About mid-table in runtime and in accuracy;

  • Results

n No differences between colour spaces

  • 4. Fudan University
slide-34
SLIDE 34

TRECVID 2005 34

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-35
SLIDE 35

TRECVID 2005 35

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-36
SLIDE 36

TRECVID 2005 36 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-37
SLIDE 37

TRECVID 2005 37

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-38
SLIDE 38

TRECVID 2005 38

  • Approach

n Builds upon previous years with intermediate visual features derived from low-level image features for pairwise frame similarities over local and longer-distances; n Used as input to a kNN classifier; n Added information-theoretic secondary feature selection to select features used in classifier;

  • Features

n Feature selection/reduction yielded improved performances;

  • Performance

n Not as good as expected because sensitive to training data;

  • Results
  • 5. FX Palo Alto Laboratory
slide-39
SLIDE 39

TRECVID 2005 39

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-40
SLIDE 40

TRECVID 2005 40

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-41
SLIDE 41

TRECVID 2005 41 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-42
SLIDE 42

TRECVID 2005 42

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-43
SLIDE 43

TRECVID 2005 43

  • Approach

n Compute frame-frame similarities over different distances and generate distance map; n Distance maps have characteristics for cuts, GTs, flashes, etc.

  • Performance

n Computation is about real-time;

  • Results
  • 6. Hong Kong Polytechnical University
slide-44
SLIDE 44

TRECVID 2005 44

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-45
SLIDE 45

TRECVID 2005 45

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-46
SLIDE 46

TRECVID 2005 46 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-47
SLIDE 47

TRECVID 2005 47

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-48
SLIDE 48

TRECVID 2005 48

  • Approach

n Builds upon previous CueVideo work at TRECVid, system is the same as 2005, except … n Noticed that GOP I/P- frame patterns (no B-frames) in TRECVid 2005 video encoding had no B-frames; n Used a different video decoder to overcome colour errors;

  • Performance

n Switching the video decoder yielded improved performances;

  • Results
  • 7. IBM Research
slide-49
SLIDE 49

TRECVID 2005 49

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-50
SLIDE 50

TRECVID 2005 50

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-51
SLIDE 51

TRECVID 2005 51 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-52
SLIDE 52

TRECVID 2005 52

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-53
SLIDE 53

TRECVID 2005 53

  • Approach

n Same as previous TRECVid submissions;

  • Features

n Exploits frame-frame differences based on colour histogram comparisons

  • Results
  • 8. Imperial College London
slide-54
SLIDE 54

TRECVID 2005 54

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-55
SLIDE 55

TRECVID 2005 55

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-56
SLIDE 56

TRECVID 2005 56 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-57
SLIDE 57

TRECVID 2005 57

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-58
SLIDE 58

TRECVID 2005 58

  • Approach

n Addressed false positives caused by abnormal lighting (flashes, reflections, camera movements, explosions, fire, etc.)

  • Features

n 2-pass algorithm - firstly compute similarity between adjacent frames using wavelets, then focus on candidate areas to eliminate false positives;

  • Performance

n Computation about real-time;

  • Results

n Submitted only 1 run, focus on hard cuts only;

  • 9. Indian Institute of Technology
slide-59
SLIDE 59

TRECVID 2005 59

Cuts

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-60
SLIDE 60

TRECVID 2005 60 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-61
SLIDE 61

TRECVID 2005 61

  • Approach

n Late arrival of paper - available at registration desk; n Compressed domain - hence fast; n Luminance adaptive threshold and image cropping equals goo results; n Last year worked in the compressed domain, extending an approach by adding edge features from DC image, colour layout, and SVM learning;

  • Results

n Worth looking at …

  • 10. KDDI R&D Laboratories, Inc.
slide-62
SLIDE 62

TRECVID 2005 62

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-63
SLIDE 63

TRECVID 2005 63

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-64
SLIDE 64

TRECVID 2005 64 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-65
SLIDE 65

TRECVID 2005 65

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-66
SLIDE 66

TRECVID 2005 66

  • Approach

n Last year worked in compressed domain, computing motion and frame statistics, then measure similarity between compensated adjacent I-frames; n This year most effort in camera motion task but submitted SBD runs based on this

  • Performance

n Good on hard cuts, and fast, not good on GTs

  • Results
  • 11. LaBRI
slide-67
SLIDE 67

TRECVID 2005 67

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-68
SLIDE 68

TRECVID 2005 68 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-69
SLIDE 69

TRECVID 2005 69

  • Approach

n Didn’t submit a paper so we don’t know !

  • Results

n Fast execution but don’t appear in the zoomed areas of graphs except for …

  • 12. Motorola Multimedia Research Laboratory
slide-70
SLIDE 70

TRECVID 2005 70

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-71
SLIDE 71

TRECVID 2005 71

  • Approach

n Late paper submitted and it doesn’t reveal much … “Video analysis + machine learning: - New to TRECVID - Developers-

  • Drs Zhenghua (Jack) Yu, SVN Vishwanathan and Alex

Smola”

  • Results

n Expensive computation but worth a peek at …

  • 13. National ICT Australia
slide-72
SLIDE 72

TRECVID 2005 72

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-73
SLIDE 73

TRECVID 2005 73

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-74
SLIDE 74

TRECVID 2005 74 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-75
SLIDE 75

TRECVID 2005 75

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-76
SLIDE 76

TRECVID 2005 76

  • Approach

n New implementation of their sliding query window approach, compute frame similarities among X frames before/after; n Frame similarities based on colour histograms; n Experimented with different (HSV) colour histogram representations;

  • Features

n Feature selection/reduction yielded improved performances;

  • Performance

n Not as good as expected because sensitive to training data;

  • Results
  • 14. RMIT University
slide-77
SLIDE 77

TRECVID 2005 77

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-78
SLIDE 78

TRECVID 2005 78

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-79
SLIDE 79

TRECVID 2005 79 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-80
SLIDE 80

TRECVID 2005 80

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-81
SLIDE 81

TRECVID 2005 81

  • Approach

n Represents video as spatio-temporal video data blocks and extracts patterns from these to indicate cuts and GTs;

  • Performance

n Efficient, expect to include camera motion information in future development;

  • Results
  • 15. Technical University of Delft
slide-82
SLIDE 82

TRECVID 2005 82

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-83
SLIDE 83

TRECVID 2005 83

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-84
SLIDE 84

TRECVID 2005 84 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-85
SLIDE 85

TRECVID 2005 85

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-86
SLIDE 86

TRECVID 2005 86

  • Approach

n Re-implement previous years very successful approaches which had evolved to a set of collaboration rules for various detectors; n Now a unified framework with SVMs combining fade-in/out detectors, GT detector and cut detectors, each developed in previous years;

  • Features

n Appears to be a mixture of different detectors;

  • Performance

n Despite individual detectors performing separately, very fast;

  • Results
  • 16. Tsinghua University
slide-87
SLIDE 87

TRECVID 2005 87

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-88
SLIDE 88

TRECVID 2005 88

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-89
SLIDE 89

TRECVID 2005 89 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-90
SLIDE 90

TRECVID 2005 90

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-91
SLIDE 91

TRECVID 2005 91

  • Approach

n Frame-frame distances computed based on pixels, and based

  • n histograms;

n Examined frame difference behaviours over time to see if it corresponds to a linear transformation;

  • Features

n Work carried out by U Modena;

  • Performance

n Could be speeded up but no optimisation;

  • Results
  • 17. University of Central Florida/U. Modena
slide-92
SLIDE 92

TRECVID 2005 92

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-93
SLIDE 93

TRECVID 2005 93

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-94
SLIDE 94

TRECVID 2005 94 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-95
SLIDE 95

TRECVID 2005 95

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-96
SLIDE 96

TRECVID 2005 96

  • Approach

n Builds upon previous years with a cut detection followed by GT detection; n Frame similarities based on colour histograms, on aggregated pixel distances and on edges;

  • Performance

n Still some issues of combining GT and cut logic detection, not appearing in zoomed areas of graphs;

  • Results
  • 18. University of Iowa
slide-97
SLIDE 97

TRECVID 2005 97 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-98
SLIDE 98

TRECVID 2005 98

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-99
SLIDE 99

TRECVID 2005 99

  • Approach

n Frame similarities measured by motion-compensated pixel differences and histogram differences for several frame distances; n An unsupervised ensemble of classifiers is then used.

  • Features

n SVM classifiers trained on 2004 data;

  • Performance

n Surprisingly efficient and good performance;

  • Results
  • 19. University of Marburg
slide-100
SLIDE 100

TRECVID 2005 100

Cuts (zoomed again)

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-101
SLIDE 101

TRECVID 2005 101

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-102
SLIDE 102

TRECVID 2005 102 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-103
SLIDE 103

TRECVID 2005 103

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-104
SLIDE 104

TRECVID 2005 104

  • Approach

n Concentrated on cut detection by shape and by a combination

  • f shape and colour features;

n Shape used Zernike moments, colour used histograms from last year; n Combination methods used various logical combinations

  • Performance

n Did well on precision for cuts, not in zoomed areas otherwise;

  • Results
  • 20. University Rey Juan Carlos
slide-105
SLIDE 105

TRECVID 2005 105

Cuts

  • CLIPS

Fudan FIU FXPal HKPU IBM IITB Im perial KDDI LaBri M arburg Motorola NICTA RM IT Tsinghua TUDelft Uiowa Um

  • dena

URJC USP CityU-HK

slide-106
SLIDE 106

TRECVID 2005 106 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-107
SLIDE 107

TRECVID 2005 107

  • Approach

n No paper submitted - again - so we don’t know

  • Results

n Appears to be fast and appearing in the zoomed areas of the graphs;

  • 21. Universidade São Paulo
slide-108
SLIDE 108

TRECVID 2005 108

Gradual transitions (zoomed)

  • !

" #

  • $
  • %&
slide-109
SLIDE 109

TRECVID 2005 109 50000 100000 150000 200000 250000 300000 350000

KDDI Marburg LaBRI USP Tsinghua IBM Motorola RMIT TUDelft Imperial Uiowa Fudan HKPU IITB REAL TIME ==> UniMore FXPal NICTA CLIPS FIU URJC CItyU-HK &" & /-&! /1

Mean runtime in seconds

slide-110
SLIDE 110

TRECVID 2005 110

Gradual transitions: Frame-P & R (zoomed)

  • !

" #

  • $
  • %&
slide-111
SLIDE 111

TRECVID 2005 111

Observations

  • Last year we said:

n Strong interest;

  • … this remains true … more in SBD than in search in

TRECVid2005 … regulars, devotees, and new participants;

n Novel approaches continue to emerge;

  • … absolutely true still … new things still being tried;

n Adding computation cost was a good idea;

  • … and it remains an interesting & important criterion;

n Lots of data available to do a more comprehensive comparative analysis;

  • … though nobody has done this yet;
slide-112
SLIDE 112

TRECVID 2005 112

Conclusions

  • What did we learn this year ?

n More new and faster methods, and data didn’t throw us any surprises, though maybe its quite similar to 2004/3 and NASA data didn’t pollute it enough ?

  • Some people ask why bother … isn’t SBD a solved problem ?

n Hard to argue against this when we can show excellent accuracy in a fraction of real-time for cuts - for GTs do we need better performance n Yet new approaches emerge each year, its very economical to run the task, and teams can break into video manipulation; n More groups do SBD than all search tasks combined !