Getting started with the 12.03.2012 3D Photography class spring - PowerPoint PPT Presentation

Getting started with the 12.03.2012 3D Photography class spring 2012 Institute of Visual Computing

Why a point cloud library? • RGB-D sensors  easy & cheap capturing of 3D data • 3D perception still a big challenges: e.g. – Object detection and recognition, tracking – Model building, mapping and localization Goal: • Provide core functionality and state-of-the-art algorithms • Iterate and develop fast  open source community Institute of Visual Computing

What can PCL do for you? • Simple interface to Kinect (wrapper around openNI) • Denoising, filtering, clustering, segmentation • Registration (ICP), model fitting, model building, mapping • 3D features, object detection, tracking, etc. Institute of Visual Computing

The Kinect RGB-D sensor • Active depth sensor • 1 infrared projector and 1 infrared camera (stereo setup) • 1 RGB camera • Camera intrinsicts are stored on the device, used by openNI  no need for calibration! • We also have some mobile Kinects, which are battery powered. Institute of Visual Computing

Example • Data capturing: provided sample code http://www.cvg.ethz.ch/teaching/2012spring/3dphoto/projects.php – (colored) point cloud – RGB image – Depth image • RGB image and depth-map already registred to each other! • Depends on PCL and OpenCV Institute of Visual Computing

Installation Documentation: http://pointclouds.org/downloads/ (1) Install prebuild libraries – sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl sudo apt-get update sudo apt-get install libpcl-all – Will also install all dependencies, which are boost (>=1.46), vtk (>=5.6), eigen (>=3.0), openni (>=1.3), flann (>=1.7.1) – Windows: all-in-one package including all dependencies (2) Compile source – Will get you the most up-to-date code and bugfixes – Use cmake to generate the project, ccmake for project configuration – See: http://www.pointclouds.org/downloads/source.html For GPU support (e.g. Kinect Fusion) CUDA Toolkit >= 4.0 required Institute of Visual Computing

Templates • PCL is a heavily templated library • E.g. allows to have one point cloud class for all point types template <typename PointT > class PointCloud { … }; usage: Implicite template pcl::PointCloud<pcl::PointXYZRGB> cloud; instantiation possible • Templated (member) functions: Usage: template <typename Type > A obj_A; void foo1 (const Type &obj); foo1 (obj_A); or B<A> obj_B; template <typename Type > void foo2 (const MyObject< Type > &obj); foo2<A> (obj_B); Explicite template instantiation necessary Institute of Visual Computing

boost::shared_ptr • Shared/smart pointers are objects which store pointers to dynamically allocated (heap) objects. • No need to delete dynamically allocated objects; reference counter takes care! • Usage: boost::shared_ptr<Object> obj_ptr (new Object()); • Used for point clouds, e.g. within class pcl::PointCloud typedef boost::shared_ptr<pcl::PointCloud<PointT> > Ptr; • Combination of templates and smart pointers: template <typename PointT > ... typename pcl::PointCloud< PointT >::Ptr cloud_ptr (new pcl::PointCloud< PointT >); Institute of Visual Computing

Tools & Troubleshooting (1) Recording datasets: – Provided sample project data_capture – http://www.openni.org/Downloads/OpenNIModules.aspx NiViewer in OpenNI Binaries to record *.ONI files (audio + video) (2) Tutorials: http://www.pointclouds.org/documentation/tutorials/ (3) API: http://docs.pointclouds.org/ (4)PCL mailing list and forum(!): http://www.pcl-users.org/ (5) PCL Blog: http://www.pointclouds.org/blog – Documentation of current code sprints (3month projects) Institute of Visual Computing

Usefull Libraries • Boost: http://www.boost.org/, lots of useful libraries • Eigen: http://eigen.tuxfamily.org, linear algebra • OpenCV: http://opencv.willowgarage.com, image processing • Structure from Motion: – Bundler: http://phototour.cs.washington.edu/bundler/ – Sparse Bunlde Adjustment: http://www.ics.forth.gr/~lourakis/sba/ • Feature detection: – SiftGPU: http://cs.unc.edu/~ccwu/siftgpu/ – CUDA Surf: http://www.d2.mpi-inf.mpg.de/surf – VLFeat: http://www.vlfeat.org/ , SIFT, MSER, k-means, etc. Institute of Visual Computing

Good luck with your projects! • If you get stuck in your project, contact us. • For coding problems, try to prepare a minimal code sample which resembels the problem/bug. Institute of Visual Computing