Ext xtraction for Point Clo loud Regis istration M. Saleh, S. - - PowerPoint PPT Presentation

▶

Jun 02, 2023 223 likes •422 views

Graphite: GRAPH-Induced feaTure Ext xtraction for Point Clo loud Regis istration M. Saleh, S. Dehghani, B. Busam, N. Navab, F. Tombari 3DV 2020 Point clouds Qi, Charles R., et al. "Pointnet: Deep learning on point sets for 3d

SLIDE 1

Graphite: GRAPH-Induced feaTure Ext xtraction for Point Clo loud Regis istration

M. Saleh, S. Dehghani, B. Busam, N. Navab, F. Tombari

3DV 2020

SLIDE 2

Point clouds

Image from Qi, Charles Ruizhongtai, et al. "Pointnet++: Deep hierarchical feature learning on point sets in a metric space." Advances in neural information processing systems. 2017. Qi, Charles R., et al. "Pointnet: Deep learning on point sets for 3d classification and segmentation." Proceedings of the IEEE conference on computer vision and pattern

recognition. 2017.

SLIDE 3

Global methods

Aoki, Yasuhiro, et al. "Pointnetlk: Robust & efficient point cloud registration using pointnet." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. Yang, Jiaolong, et al. "Go-ICP: A globally optimal solution to 3D ICP point-set registration." IEEE transactions on pattern analysis and machine intelligence 38.11 (2015): 2241-2254.

SLIDE 4

Local methods

Deng, Haowen, Tolga Birdal, and Slobodan Ilic. "Ppf- foldnet: Unsupervised learning of rotation invariant 3d local descriptors." Proceedings of the European Conference on Computer Vision (ECCV). 2018. Zeng, Andy, et al. "3dmatch: Learning local geometric descriptors from rgb-d reconstructions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.

SLIDE 5

Graphite

Sparse description Dense description and keypoint per patch

SLIDE 6

Graphite

voxel representation point representation graph representation Descriptor

SLIDE 7

𝑞3 𝑞2 𝑞1 𝑞4 𝑞5 𝑞6 𝑞7 𝑞8 𝑞9 𝑞10 𝑓(1,2) 𝑞𝑗 = (𝑦𝑗, 𝑧𝑗, 𝑨𝑗, 𝑏𝑗, 𝑐𝑗, 𝑑𝑗) 𝑓(𝑘, 𝑙) = ቐ 𝑠 𝑠 + ||𝑞𝑘 − 𝑞𝑙|| , 𝑗𝑔||𝑞𝑘 − 𝑞𝑙|| < 𝑠 , 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓

Graph construction

SLIDE 8

𝑳 = 𝟐 𝑳 = 𝟑 𝑳 = 𝟒 GCN K=1 (6,8)

Input Patch

𝐵[𝑜𝑦𝑜] 𝑌[𝑜𝑦6] 𝑍[𝑜𝑦1]

GCN K=2 (8,16) GCN K=3 (16,32) GCN K=3 (32,16) GCN K=2 (16,8) GCN K=1 (8,1) scatter max FC

𝑇[1] 𝐸[1𝑦32]

scatter max FC

Architecture

𝑌𝑗

′ = ෍ 𝑙=0 𝐿

𝐸′−1/2𝐵𝑙𝐸′−1/2𝑌𝑗Θ𝑙

SLIDE 9

Training datasets

Positive Anchor Negative ModelNet40 registration pair and patches

SLIDE 10

Initialization

Graphite

Triplet loss Loss Loss Loss

𝐸𝑠 𝐸𝑞 𝐸𝑜

Graphite Graphite

𝑀𝑡 = (𝑍 − ෠ 𝑍)2 𝑀𝑇 = (𝑇 − መ 𝑇)2 𝑀𝐸 = |𝐸𝑠 − 𝐸𝑞| 𝐸𝑠 − 𝐸𝑞 + 𝑛. |𝐸𝑠 − 𝐸𝑜|

SLIDE 11

Training with pose variations

Graphite

Triplet Descriptor loss Triplet Detection loss

𝐸𝑠 𝐸𝑞 𝐸𝑜

Graphite Graphite

Relative pose

warping

SLIDE 12

Original pair Keypoint matches Graphite Graphite + ICP

ModelNet40 registration

SLIDE 13

ModelNet40 registration

SLIDE 14

Registration under noise

SLIDE 15

3dmatch registration

Descriptor visualisation 3DMatch seed points Keypoint scores Validated keypoints

SLIDE 16

3dmatch registration

Scene 1 + Keypoints Registered with Graphite Scene 2 + Keypoints

SLIDE 17

Geometric registration benchmark

SLIDE 18

Conclusion

We describe patches using our lightweight graph-based model
Our model is trained with minimum supervision
It also extracts salient interest points, keypoints
Keypoints can be used to downsample the original point cloud to a

uniformly distributed set

For registration a combination of keypoint and descriptor can be used
Although trained on synthetic point clouds, our model generalizes

well with real depth scans

SLIDE 19

Graphite: GRAPH-Induced feaTure Ext xtraction for Point Clo loud Regis istration

Paper ID 2 Official Code and pretrained models github.com/mahdi-slh/Graphite