Index Index � Introduction Shape Context and Chamfer Shape Context and Chamfer � Previous Work � Shape context matching Matching in Cluttered Scenes Matching in Cluttered Scenes � Chamfer matching � Problems with shape context � Solutions to the problems � Edge orientation Presented by Jong Taek Lee � Figural continuity � Results of hand tracking and word recognition � Discussion & Conclusion Introduction Introduction Previous Work Previous Work � How to detect a hand? � Shape Context [Belongie et al ., 00] – Invariance to translation and scale – High performance in � Digit recognition : MNIST dataset � Silhouettes : MPEG-7 database � Common household objects: COIL-20 database � Comparison of matching methods � Shape context vs. Chamfer matching � Chamfer Matching [Barrow et al ., 77] − efficient hierarchical matching [Borgefors, 88] � Enhancements for shape context − pedestrian detection [Gavrila, 00] � Robustness to clutter Shape Context: Histogram Shape Context: Histogram Shape Context: Matching Shape Context: Matching Cost Matrix C i j Image Points Template Points � Shape context of a point: log-polar histogram of C χ 2 Test ij the relative positions of all other points Optimal Correspondence φ � Similar points on shapes opt have similar histograms Cost Function Bi-partite Graph ∑ = C C Matching , φ sc i ( i ) i 1
Shape- -Context: Matching Context: Matching Scale Invariance in Clutter ? Shape Scale Invariance in Clutter ? � Median of pairwise point distances is used as scale factor 50.5 41.6 � Clutter will affect this scale factor 50.5 121.9 Scale Invariant in Clutter ? Scale Invariant in Clutter ? Multiple Edge Orientations Multiple Edge Orientations � Significant clutter � Edge pixels are divided into 8 groups – Unreliable scale factor based on orientation – Incorrect correspondences � Shape contexts are � Solution calculated separately for each group – Calculate shape contexts at different scales and match at different scales � Total matching score – Computationally expensive is obtained by adding individual χ 2 scores Single vs. Multiple Orientation Single vs. Multiple Orientation No Figural Continuity No Figural Continuity � No continuity constraint � Adjacent points in one shape are matched to distant points in the other 2
Imposing Figural Continuity Imposing Figural Continuity Imposing Figural Continuity Imposing Figural Continuity v φ (i) u i v φ (i-1) u i-1 u i-2 v φ (i-2) � u i and u i-1 are neighboring points on the model shape u v φ (i) u i � φ is the correspondence between two shape points v φ (i-1) u i-1 � Corresponding points v φ (i) and v φ (i-1) need to be neighboring points on target shape v u i-2 v φ (i-2) Imposing Figural Continuity Imposing Figural Continuity Viterbi Algorithm Viterbi Algorithm � Initialization � Minimize the cost function for φ � Propagation – Compute and sum up cost � Ordering of the model shape is known – Store a pointer � Termination � Use Viterbi Algorithm � Optimal Path Backtracking With Figural Continuity With Figural Continuity With Figural Continuity With Figural Continuity Similar Shapes Different Scale 3
With Figural Continuity With Figural Continuity With Figural Continuity With Figural Continuity Small Rotation Shape Variation With Figural Continuity With Figural Continuity Chamfer Matching Chamfer Matching Clutter � Matching technique cost is integral along contour � Distance transform of the Canny edge map Distance Transform Distance Transform Chamfer Matching Chamfer Matching (x,y) (x,y) d d � Chamfer score is average nearest distance from template points to image points � Distance image gives the distance to the nearest � Nearest distances are readily obtained from the edge at every pixel in the image distance image � Calculated only once for each frame � Computationally inexpensive 4
Chamfer Matching Chamfer Matching Chamfer Matching Chamfer Matching � Distance image provides a smooth cost function � Efficient searching techniques can be used to find correct template Chamfer Matching Chamfer Matching Chamfer Matching Chamfer Matching Chamfer Matching Chamfer Matching Chamfer Matching Chamfer Matching 5
Multiple Edge Orientations Applications: Hand Detection Multiple Edge Orientations Applications: Hand Detection � Similar to Gavrila, edge pixels are divided into 8 � Initializing a hand model for tracking groups based on orientation – Locate the hand in the image – Adapt model parameters � Distance transforms are calculated separately for – No skin color information used each group – Hand is open and roughly front-parallel � Total matching score is obtained by adding individual chamfer scores Results: Hand Detection Results: Hand Detection Results: Hand Detection Results: Hand Detection Shape Context with Shape Context with Original Shape Context Chamfer Matching Original Shape Context Chamfer Matching Continuity Constraint Continuity Constraint Applications: CAPTCHA Applications: CAPTCHA EZ- EZ -Gimpy results Gimpy results Chamfer cost for each letter template � Completely Automated Public Turing test to tell Computers and Humans Apart [Blum et al. , 02] � Used in e-mail sign up for Yahoo accounts Word matching cost: average chamfer cost � Word recognition with shape variation and added noise + variance of distances Examples: Top 3 matches (dictionary 561 words) right 25.34 fight 27.88 night 28.42 89.5% correct matches using 1 template per letter 93.2% correct matches using 2 templates per letter Shape context 82.7% [Mori & Malik, 03] 6
Discussion Discussion Discussion Discussion � The original shape context matching � Shape Context with Continuity Constraint – Not invariant in clutter – Includes contour continuity & curvature – Iterative matching is used in the original shape – Robust to substantial amount of clutter context paper – Much better correspondences and model – Correct point correspondence in the initial alignment just from initial matching matching is quite small in substantial clutter – No need for iteration – Iterative matching will not improve the – More robust to small variations in scale, performance rotation and shape. Discussion Discussion Conclusion Conclusion � Chamfer Matching Use shape context when – Variant to scale and rotation – There is not much clutter – More sensitive to small shape changes than – There are unknown shape variations from the shape context templates (e.g. two different types of fish) – Need large number of template shapes – Speed is not the priority But – Robust to clutter – Computationally cheap compared to shape context Conclusion Conclusion References References � A. Thayananthan, B. Stenger, P. H. S. Torr, and R. Cipolla. Shape Context and Chamfer Matching in Cluttered Scenes. Chamfer matching is better when CVPR 2003. – There is substantial clutter � D. Gavrila. Pedestrian Detection from a – All expected shape variations are well- Moving Vehicle. ECCV 2000 represented by the shape templates � S. Belongie, J. Malik, and J. Puzicha. – Robustness and speed are more important Shape matching and object recognition using shape contexts. PAMI 2002. The original version of this presentation file is from A. Thayananthan, B. Stenger, P. H. S. Torr, and R. Cipolla 7
Thank You! 8
Recommend
More recommend
