Overview • Introduction Cooperative Object Tracking • Calibration with • Tracking Multiple PTZ Cameras • Segmentation – Target handover Ivo Everts • Conclusion PhD student at UvA Kingston University London University of Amsterdam Supervision: Theo Gevers (MSc), Nicu Sebe (PhD) Supervision: Graeme Jones ISIS / ISLA DIRC http://www.science.uva.nl/research/isla/ http://www.kingston.ac.uk/dirc/ Introduction Introduction • Current research in Visual Surveillance • Current research on Visual Surveillance – Scene understanding – Scene understanding • Sensor networks • Sensor networks – heterogeneous – heterogeneous • Advantages of PTZ cameras • Advantages of PTZ cameras – Active – Active – High resolution imaging – High resolution imaging
Introduction Introduction • Current research on Visual Surveillance • Current research on Visual Surveillance – Scene understanding – Scene understanding • Sensor networks • Sensor networks – heterogeneous – heterogeneous • Advantages of PTZ cameras • Advantages of PTZ cameras – Active – Active – High resolution imaging – High resolution imaging Introduction Introduction • Current research on Visual Surveillance • Current research on Visual Surveillance – Scene understanding – Scene understanding • Sensor networks • Sensor networks – heterogeneous – heterogeneous • Advantages of PTZ cameras • Advantages of PTZ cameras – Active – Active – High resolution imaging – High resolution imaging • Goal: PTZ tracking with target handover
Calibration Calibration • Communication about target location • Communication about target location • Cameras calibrated wrt common ground • Cameras calibrated wrt common ground plane plane Calibration Calibration • Communication about target location • Geometry! • Cameras calibrated wrt common ground plane (0,0,0)
Calibration Calibration • Geometry! • From pixels to world coordinates • H,p0,U,V resolved by least squares (p’=p+p0) Calibration Tracking • Example • Let camera move along with target • Problems with motion detection • Mean Shift – Assumed initialised – Target representation & localisation
Tracking Tracking • Let camera move along with target • Let camera move along with target • Problems with motion detection • Problems with motion detection • Mean Shift • Mean Shift – Assumed initialised – Assumed initialised – Target representation & localisation – Target representation & localisation Tracking Tracking • Mean Shift • Mean Shift – Target representation: colour histogram – Target representation: colour histogram – Target q, candidate p – Target q, candidate p – Weighted by kernel K(x) – Weighted by kernel K(x) • Profile k( ||x||² ) • Profile k( ||x||² )
Tracking Tracking • Mean Shift • Mean Shift – Target representation: colour histogram – Target representation: colour histogram – Target q, candidate p – Target q, candidate p – Weighted by kernel K(x) – Weighted by kernel K(x) • Profile k( ||x||² ) • Profile k( ||x||² ) • Epanechnikov kernel: Tracking Tracking • Candidate profile: function of new target • Candidate profile: function of new target centroid y centroid y – k( ||y-xi|| ² ) – k( ||y-xi|| ² ) • Metric between p and q function of y • Metric between p and q function of y – Bhattacharya distance – Bhattacharya distance – ( p=p(y) )
Tracking Tracking • Target localisation • Target localisation – Minimise d(p(y),q) wrt y – Minimise d(p(y),q) wrt y • New centroid y : kernel and data weighted • New centroid y : kernel and data weighted sum over pixels locations sum over pixels locations Tracking Tracking • Target localisation • PTZ tracking algorithm – Minimise d(p(y),q) wrt y • New centroid y : kernel and data weighted sum over pixels locations y0 y1
Tracking Segmentation • Example • Target handover • Statistical framework – Find target given the colour model and location estimate of the other camera Segmentation Segmentation • Target handover • Target handover • Statistical framework • Statistical framework – Find target given the colour model and – Find target given the colour model and location estimate of the other camera location estimate of the other camera • P(O|c,i) – Proportional to p(i|O)p(c|O)
Segmentation Segmentation • Classify pixels • Classify pixels • Open image • Open image • Find connected components • Find connected components • Constrain blob on size • Constrain blob on size Segmentation Segmentation • Classify pixels • Classify pixels • Open image • Open image • Find connected components • Find connected components • Constrain blob on size • Constrain blob on size
Segmentation Segmentation • Playing hide and seek – Init cam 1 – Cam1 tracks target – Cam 2 counts to 5 – Cam 2 seeks target – When found: Cam 2 tracks target – Cam 1 counts to 5 – Etcetera Conclusion The End • Successful target handover • Thank you – In real time • Simple target representation – Drawbacks • Indoor setting • Need for automation • Camera quality • Zoom • Semantics • Evaluation
Colour Colour • The problem with colour • The problem with colour • Different data acquisition processes • Different data acquisition processes • Find out how different • Find out how different • Experiments • Experiments Colour Colour • The problem with colour • The problem with colour • Different data acquisition processes • Different data acquisition processes • Find out how different • Find out how different • Experiments • Experiments
Colour Colour • The problem with colour • Remarkable result in xy colour space • Different data acquisition processes • x=X/(X+Y+Z) etc • Find out how different • Experiment: analyse displacement between peaks in quantised spaces of • Experiments both cameras Generate distributions of K patches of J colours in S colour spaces for C cameras. Analyse! Colour Colour • Remarkable result in xy colour space • Remarkable result in xy colour space • x=X/(X+Y+Z) etc • x=X/(X+Y+Z) etc • Experiment: analyse displacement • Experiment: analyse displacement between peaks in quantised spaces of between peaks in quantised spaces of both cameras both cameras
Colour Colour • Remarkable result in xy colour space • Displacement plot • x=X/(X+Y+Z) etc • Structure! • Experiment: analyse displacement between peaks in quantised spaces of both cameras Colour Colour • Displacement plot • Displacement plot • Structure! • Structure! • Compensate for it: colour calibration
Colour • Displacement plot • Structure! • Compensate for it: colour calibration • Conclusion – xy shows hardware difference
Recommend
More recommend
