Ext xtraction for Point Clo loud Regis istration M. Saleh, S. - PowerPoint PPT Presentation

Graphite: GRAPH-Induced feaTure Ext xtraction for Point Clo loud Regis istration M. Saleh, S. Dehghani, B. Busam, N. Navab, F. Tombari 3DV 2020

Point clouds Qi, Charles R., et al. "Pointnet: Deep learning on point sets for 3d classification and Image from Qi, Charles Ruizhongtai, et al. "Pointnet++: segmentation." Proceedings of the IEEE conference on computer vision and pattern Deep hierarchical feature learning on point sets in a recognition . 2017. metric space." Advances in neural information processing systems . 2017.

Global methods Aoki, Yasuhiro, et al. "Pointnetlk: Robust & efficient point cloud registration using Yang, Jiaolong, et al. "Go-ICP: A globally optimal solution to 3D ICP point-set registration." IEEE transactions on pattern analysis and machine intelligence 38.11 (2015): 2241-2254. pointnet." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2019.

Local methods Deng, Haowen, Tolga Birdal, and Slobodan Ilic. "Ppf- Zeng, Andy, et al. "3dmatch: Learning local geometric descriptors from rgb-d foldnet: Unsupervised learning of rotation invariant 3d local reconstructions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2017. descriptors." Proceedings of the European Conference on Computer Vision (ECCV) . 2018.

Graphite Dense description and Sparse description keypoint per patch

Graphite Descriptor point representation graph representation voxel representation

Graph construction 𝑞 8 𝑞 5 𝑞 4 𝑞 7 𝑞 6 𝑞 2 𝑓(1,2) 𝑞 1 𝑞 3 𝑞 9 𝑞 10 𝑞 𝑗 = (𝑦 𝑗 , 𝑧 𝑗 , 𝑨 𝑗 , 𝑏 𝑗 , 𝑐 𝑗 , 𝑑 𝑗 ) 𝑠 𝑠 + ||𝑞 𝑘 − 𝑞 𝑙 || , 𝑗𝑔||𝑞 𝑘 − 𝑞 𝑙 || < 𝑠 𝑓(𝑘, 𝑙) = ቐ 0 , 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓

Architecture 𝑳 = 𝟑 𝑳 = 𝟒 𝑳 = 𝟐 GCN GCN Input Patch GCN K=1 K=2 GCN K=3 𝑍 [𝑜𝑦1] (8,1) GCN (16,8) 𝑌 [𝑜𝑦6] GCN K=3 (32,16) K=2 K=1 𝑇 [1] (16,32) (8,16) scatter max FC 𝐵 [𝑜𝑦𝑜] (6,8) FC 𝐸 [1𝑦32] scatter max 𝐿 ′ = ෍ 𝐸 ′−1/2 𝐵 𝑙 𝐸 ′−1/2 𝑌 𝑗 Θ 𝑙 𝑌 𝑗 𝑙=0

Training datasets Anchor Positive Negative ModelNet40 registration pair and patches

Initialization Loss Graphite 𝑀 𝑡 = (𝑍 − ෠ 𝑍) 2 𝐸 𝑠 Loss 𝑀 𝑇 = (𝑇 − መ 𝑇) 2 Graphite 𝐸 𝑞 Triplet loss |𝐸 𝑠 − 𝐸 𝑞 | 𝑀 𝐸 = 𝐸 𝑠 − 𝐸 𝑞 + 𝑛. |𝐸 𝑠 − 𝐸 𝑜 | Loss Graphite 𝐸 𝑜

Training with pose variations Relative pose warping Graphite 𝐸 𝑠 Triplet Detection loss Graphite 𝐸 𝑞 Triplet Descriptor loss Graphite 𝐸 𝑜

ModelNet40 registration Original pair Keypoint matches Graphite Graphite + ICP

ModelNet40 registration

Registration under noise

3dmatch registration Keypoint scores Validated keypoints Descriptor visualisation 3DMatch seed points

3dmatch registration Scene 2 + Keypoints Registered with Graphite Scene 1 + Keypoints

Geometric registration benchmark

Conclusion • We describe patches using our lightweight graph-based model • Our model is trained with minimum supervision • It also extracts salient interest points, keypoints • Keypoints can be used to downsample the original point cloud to a uniformly distributed set • For registration a combination of keypoint and descriptor can be used • Although trained on synthetic point clouds, our model generalizes well with real depth scans

Graphite: GRAPH-Induced feaTure Ext xtraction for Point Clo loud Regis istration Paper ID 2 Official Code and pretrained models github.com/mahdi-slh/Graphite