Getting started with the 12.03.2012 3D Photography class spring - - PowerPoint PPT Presentation

Institute of Visual Computing

Getting started with the

12.03.2012 3D Photography class spring 2012

Institute of Visual Computing

Why a point cloud library?

• RGB-D sensors  easy & cheap capturing of 3D data
• 3D perception still a big challenges: e.g.

– Object detection and recognition, tracking – Model building, mapping and localization

Goal:

• Provide core functionality and state-of-the-art algorithms
• Iterate and develop fast  open source community

Institute of Visual Computing

What can PCL do for you?

• Simple interface to Kinect (wrapper around openNI)
• Denoising, filtering, clustering, segmentation
• Registration (ICP), model fitting, model building, mapping
• 3D features, object detection, tracking, etc.

Institute of Visual Computing

The Kinect RGB-D sensor

• Active depth sensor
• 1 infrared projector and 1 infrared camera (stereo setup)
• 1 RGB camera
• Camera intrinsicts are stored
• n the device, used by openNI

 no need for calibration!

• We also have some mobile Kinects, which are battery powered.

Institute of Visual Computing

Example

• Data capturing: provided sample code

http://www.cvg.ethz.ch/teaching/2012spring/3dphoto/projects.php – (colored) point cloud – RGB image – Depth image

• RGB image and depth-map already registred to each other!
• Depends on PCL and OpenCV

Institute of Visual Computing

Installation

Documentation: http://pointclouds.org/downloads/ (1) Install prebuild libraries

– sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl sudo apt-get update sudo apt-get install libpcl-all

– Will also install all dependencies, which are boost (>=1.46), vtk (>=5.6), eigen (>=3.0), openni (>=1.3), flann (>=1.7.1) – Windows: all-in-one package including all dependencies

(2) Compile source

– Will get you the most up-to-date code and bugfixes – Use cmake to generate the project, ccmake for project configuration – See: http://www.pointclouds.org/downloads/source.html

For GPU support (e.g. Kinect Fusion) CUDA Toolkit >= 4.0 required

Institute of Visual Computing

Templates

• PCL is a heavily templated library
• E.g. allows to have one point cloud class for all point types

template <typename PointT> class PointCloud { … }; usage: pcl::PointCloud<pcl::PointXYZRGB> cloud;

• Templated (member) functions:

template <typename Type> void foo1 (const Type &obj);

• r

template <typename Type> void foo2 (const MyObject<Type> &obj);

Usage:

A obj_A; foo1 (obj_A); B<A> obj_B; foo2<A> (obj_B);

Implicite template instantiation possible Explicite template instantiation necessary

Institute of Visual Computing

boost::shared_ptr

• Shared/smart pointers are objects which store pointers to

dynamically allocated (heap) objects.

• No need to delete dynamically allocated objects; reference

counter takes care!

• Usage: boost::shared_ptr<Object> obj_ptr (new Object());
• Used for point clouds, e.g. within class pcl::PointCloud

typedef boost::shared_ptr<pcl::PointCloud<PointT> > Ptr;

• Combination of templates and smart pointers:

template <typename PointT> ... typename pcl::PointCloud<PointT>::Ptr cloud_ptr (new pcl::PointCloud<PointT>);

Institute of Visual Computing

Tools & Troubleshooting

(1) Recording datasets:

– Provided sample project data_capture – http://www.openni.org/Downloads/OpenNIModules.aspx NiViewer in OpenNI Binaries to record *.ONI files (audio + video)

(2) Tutorials: http://www.pointclouds.org/documentation/tutorials/ (3) API: http://docs.pointclouds.org/ (4)PCL mailing list and forum(!): http://www.pcl-users.org/ (5) PCL Blog: http://www.pointclouds.org/blog

– Documentation of current code sprints (3month projects)

Institute of Visual Computing

Usefull Libraries

• Boost: http://www.boost.org/, lots of useful libraries
• Eigen: http://eigen.tuxfamily.org, linear algebra
• OpenCV: http://opencv.willowgarage.com, image processing
• Structure from Motion:

– Bundler: http://phototour.cs.washington.edu/bundler/ – Sparse Bunlde Adjustment: http://www.ics.forth.gr/~lourakis/sba/

• Feature detection:

– SiftGPU: http://cs.unc.edu/~ccwu/siftgpu/ – CUDA Surf: http://www.d2.mpi-inf.mpg.de/surf – VLFeat: http://www.vlfeat.org/ , SIFT, MSER, k-means, etc.

Institute of Visual Computing

Good luck with your projects!

• If you get stuck in your project, contact us.
• For coding problems, try to prepare a minimal code sample

which resembels the problem/bug.