Shot Boundary Experiments at The University of Iowa David - - PowerPoint PPT Presentation

Shot Boundary Experiments at The University of Iowa

David Eichmann1,2 & Dong-Jun Park2 1School of Library and Information Science 2Computer Science Department

Basic Assumptions

• A relatively small number of ‘basic’ metrics can be

composed into a metric that can perform well

• Observed with ASR (e.g., Rover)
• For this year, focus on localized video measures
• i.e., contiguous pairs of frames

Basic Metrics

• Color Histogram Similarity
• pixels compressed to a 9-bit color scheme,

yielding a 512-bin histogram

• Frame Color Distance
• scale frames to 60 x 60 thumbnails and then

average the color space distance of all pixel pairs

• Frame Edge Distance
• generate an edge representation of frames and

then the percentage of entry and exit edges

A Sample Image

A Sample Image

• Boolean Predicate of Basic Metrics
• Composite-1: h < 0.95 & (d < 0.80 | e < 0.85)
• Composite-2: (h < 0.82 & d < 0.82)

| (h < 0.79 & e < 0.79)

• Product of Basic Metrics
• d * e * h < 0.60

Composite Metrics

Tuning / Visualization

!" !"#$ !"#% !"#& !"#' !( !)"" !)$" !)%" !)&" !)'" !%"" !%$" !%%" !%&" !%'" !*"" +,-,./0,12 3,-4!564789:6; (<<'"("%=>?@#-AB :,61/974 4:B4 C,618B0/- 78-A86,14D(!E8F9:/0,46 A08:F71!E8F9:/0,46 1461!641!E8F9:/0,46

Tuning / Visualization

!" !"#$ !"#% !"#& !"#' !( !" !$" !%" !&" !'" !("" !($" !(%" !(&" !('" !$"" )*+*,-.*/0 1*+2!342567849 (::'"$(";<==#+>? 8*4/-752 28?2 @*4/6?.-+ 56+>64*/2A(!B6C78-.*24 >.68C5/!B6C78-.*24 /24/!42/!B6C78-.*24

Official Runs

Run Metric All Cuts Gradual Rec Prec Rec Prec Rec Prec F- Rec F- Prec

UIowaSB0301 histo. 0.445 0.804 0.554 0.937 0.178 0.389 0.234 0.960 UIowaSB0302 dist. 0.607 0.855 0.835 0.963 0.051 0.158 0.178 0.826 UIowaSB0303 comp-1 0.657 0.785 0.810 0.948 0.285 0.360 0.274 0.907 UIowaSB0304 prod. 0.722 0.785 0.893 0.976 0.306 0.330 0.300 0.938 UIowaSB0305 comp-2 0.665 0.432 0.772 0.957 0.406 0.123 0.286 0.777

Shot Boundaries, Overall Results

!" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( )*+,-.-/0 1+,233 4-.5/6*27 8-.520,+ ,/79/.-5+:( 9*/8;,5 ,/79/.-5+:$

Shot Boundaries, Cut T ransitions

!" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( )*+,-.-/0 1+,233 4-.5/6*27 8-.520,+ ,/79/.-5+:( 9*/8;,5 ,/79/.-5+:$

Shot Boundaries, Gradual T ransitions

!" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( )*+,-.-/0 1+,233 4-.5/6*27 8-.520,+ ,/79/.-5+:( 9*/8;,5 ,/79/.-5+:$

Shot Boundaries, By T ransition Type & Source

!" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( )*+,-.-/0 1+,233 456!7*20.-6-/0.8!92.-,!:+6;/<. =94!;-.6/>*2? =94!<-.620,+ 4@@!;-.6/>*2? 4@@!<-.620,+ !" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( )*+,-.-/0 1+,233 456!7*20.-6-/0.8!4/9:/.-6+!;+6</=. >?4!,/9:/.-6+@( >?4!:*/=5,6 >?4!,/9:/.-6+@$ 4AA!,/9:/.-6+@( 4AA!:*/=5,6 4AA!,/9:/.-6+@$

Shot Boundaries, By T ransition Type & Source

!" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( )*+,-.-/0 1+,233 4*25623!7*20.-8-/0.9!:2.-,!;+8</5. =:>!<-.8/?*2@ =:>!5-.820,+ >AA!<-.8/?*2@ >AA!5-.820,+ !" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( )*+,-.-/0 1+,233 4*25623!7*20.-8-/0.9!:/;</.-8+!=+8>/5. ?@:!,/;</.-8+A( ?@:!<*/56,8 ?@:!,/;</.-8+A$ :BB!,/;</.-8+A( :BB!<*/56,8 :BB!,/;</.-8+A$

• Basic metrics can perform surprisingly well on

cuts

• Composite metrics can damp out peculiarities of

component metrics, just as in ASR

• Product metrics appear to be the way to go
• No arcania of boolean exploration

Conclusions

• The obvious...
• Frame sequence metrics
• Follow the approach presented here
• Specialized event detectors
• camera flash
• video effects (e.g., wipes, dissolves, ...)

Future W

• rk