ACCELERATED VISUAL TRACKING FRAMEWORK David Concha Gomez, Raul - PowerPoint PPT Presentation

DEVELOPMENT OF A GPU ACCELERATED VISUAL TRACKING FRAMEWORK David Concha Gomez, Raul Cabido Valladolid, Antonio Sanz Montemayor, Juan José Pantrigo (Universidad Rey Juan Carlos) {david.concha, raul.cabido, juanjose.pantrigo, antonio.sanz}@urjc.es

Agenda  Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

Agenda  Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

Introduction  The goal is to build a framework that eases the development of visual tracking systems  Visual tracking estimates the evolution of a system state through time  Particle Filter (PF) uses a population of N estimations  It can be computationally expensive  But it is highly parallelizable

Introduction (particle filter)  Initialization:  Create a new random population P k P 1 Particle Population P 2 P 1 =(x 1 ,y 1 , p 1 ) P 2 =(x 2 ,y 2 , p 2 ) … p i =?? P k =(x k ,y k , p k ) … P n =(x n ,y n , p n ) P n

Introduction (particle filter)  Evaluation:  Using the measurement model, each particle is weighted P k P 1 Particle Population P 2 P 1 =(x 1 ,y 1 , 3 ) P 2 =(x 2 ,y 2 , 27 ) … P k =(x k ,y k , 63 ) … P n =(x n ,y n , 4 ) P n

Introduction (particle filter)  Estimation:  Particle with the highest weight is selected as the best estimator at time step t

Introduction (particle filter)  Selection and diffusion:  A new population is generated based on the system estimate, and every particle is diffused to provide diversity t t t+1

Agenda  Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

2D Articulated Tracking  Our 2D articulated tracking problem in 2009  N segments  X =(x, y , q 1 , q 2 , q 3 ,…, q n )

2D Articulated Tracking  In each frame we know:  The previous state of the system  The blob information  The edge information

2D Articulated Tracking  Each particle defines a image regions that are measured and compacted in tiles (32x32) X =(x, y , q 1 , q 2 , q 3 , q 4 , q 5 , q 6 )

2D Articulated Tracking  Because the evaluation of each particle is independent on each other, It is highly parallelizable

2D Articulated Tracking  Reducing the compact texture results in the particles weights  Reducing all the weights give the best particle  A new population is created by a resampling technique (or any other mechanism)

2D Articulated Tracking GTX260 and 640x480 700.0 Standard PF 629.1 Fps (higher is better) 585.1 584.3 600.0 PF with Geometric Templates 500.0 PF with Geom.Temp & Elitism 400.0 284.1 260.5 258.0 300.0 200.0 94.4 79.1 79.1 100.0 0.0 256 1024 4096 # Particles

Agenda  Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

CUDA Adaptation  OpenGL + Shaders:  Good performance  Reduction: requires power of 2 tiles (i.e. 32x32)  OpenGL for rasterizer, CUDA for Reduction:  Reduction: more efficient for arbitrary tile size  Interoperability impact performance  CUDA Rasterizer and Reduction:  For high number of tiles it is as efficient as OpenGL+Shader without its drawbacks

Agenda  Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

3D Visual tracking  Calibrated and synchronized system  In a multiview system, many images simultaneously  Increased computational cost of evaluation

3D Visual tracking  3 DOF (x,y,z) to a 3D Volume (8 vertices) ( x k , y k , z k ) 8 vertices

3D Visual tracking  3D Volume (8 vertices) to an Axis Aligned Bounding Box 8 vertices AAB B For each camera x m y ,  m p  t t I ( k , l ) v , i  k , l x , y 0 0

3D Visual tracking  One thread block obtain the weight of one particle  The AABB approach simplified computation but induce some error  The remaining PF stages are similar

3D Visual tracking Performance (higher is better) CPU 1 thread FPS CPU 2 thread 450 CPU 4 thread 400 CPU 8 thread GTX 780 350 300 250 200 150 100 50 0 10 100 500 1000 # Particles

3D Visual tracking Hybrid configuration 1000 particles FPS 120 x49.7 CPU 1 thread x46.4 x46.7 100 CPU 8 threads x38.2 GPU 80 60 x21.8 40 20 x1 x3.9 0 CPU CPU+GPU(W) CPU+GPU(BS+W) GPU

3D Visual tracking GPUs Comparison FPS GTX 680 450 GTX 780 400 GTX Titan Black 350 300 250 200 150 100 50 0 10 100 500 1000 Particles Partículas

3D Visual tracking Distance (mm) Ground Truth Ground Truth vs AABB X 1500 Ground Truth Y 1000 500 0 -500 -1000 -1500 1 76 151 226 301 376 451 526 601 676 751 826 901

Agenda  Introduction  2D Articulated Tracking  CUDA Adaptation  3D Object Tracking  3D Articulated Tracking (work in progress)

3D Articulated Tracking  Articulated body model with more than 30 DOF  larger particle population  very computationally expensive evaluation +  Each particle is represented by multiple regions (articulated) in many images (3D)  Regions no longer axis aligned and cylindrical

3D Articulated Tracking  PF alone is not enough  Huge state space

Conclusions  Particle filter is a scalable and parallel friendly method.  Its weight computation is the most demanding stage  2D articulated tracking  Up to 630 fps on GPU using shaders (256 particles)  3D object tracking  Improvement of almost x50 against CPU  Good accuracy with 100 particles running at 300 fps.  Now combining both ideas for the 3D articulated problem

Questions 2004 2006 2009 SIGGRAPH poster SIGGRAPH poster First PF weighting on GPU Complete PF on GPU GPUTA 2009 2012 2D articulated tracking GTC 2013 CUDA Rasterizer Thank you for your attention GTC 3D Object Tracking 2014 (Kinect+GPU) 2014 GTC 2015-16 ?? High-Performance Multiview 3D Tracking JRTIP Using Particle Filters CPU vs GPU 3D articulated Performance study of PF