[PPT] - PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree PowerPoint Presentation

SLIDE 1

Kaichun Mo, He Wang, Xinchen Yan, Leonidas Guibas

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

(Data and Code has been released)

1

SLIDE 2

Kaichun Mo, He Wang, Xinchen Yan, Leonidas Guibas

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

(Data and Code has been released)

2

SLIDE 3

Kaichun Mo, He Wang, Xinchen Yan, Leonidas Guibas

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

(Data and Code has been released)

3

SLIDE 4

Previous Works: 3D Shape Generation

4

Wu et al., Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling, NeurIPS 2016 Achlioptas et al., Learning Representations and Generative Models for 3D Point Clouds, ICML 2018 Chen et al., Learning Implicit Fields for Generative Shape Modeling, CVPR 2019

SLIDE 5

Previous Works: Part-aware 3D Shape Generation

5

Li et al., GRASS: Generative Recursive Autoencoders for Shape Structures, Siggraph 2017 Wang et al., Global-to-Local Generative Model for 3D Shapes, Siggraph Asia 2018 Mo et al., Structurenet: Hierarchical Graph Networks for 3D Shape Generation, Siggraph Asia 2019

SLIDE 6

Previous Works: Conditional Content Generation

6

Johnson et al., Image Generation from Scene Graphs, CVPR 2018 Chen et al., Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings, ACCV 2018

SLIDE 7

Representing Shape as a Hierarchy of Parts

7

Back

Back Support Back Surface Back Single Surface

Arm Arm

Sofa Arm Sofa Arm

Seat

Seat Support Seat Frame Frame Bar Frame Bar Frame Bar Frame Bar Seat Surface Seat Single Surface

Base

Regular Leg Base Leg Leg Leg Leg

Chair

Mo et al., PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding, CVPR 2019

SLIDE 8

Symbolic Part Tree: Describing a Shape Structure

8

Back

Back Support Back Surface Back Single Surface

Arm Arm

Sofa Arm Sofa Arm

Seat

Seat Support Seat Frame Frame Bar Frame Bar Frame Bar Frame Bar Seat Surface Seat Single Surface

Base

Regular Leg Base Leg Leg Leg Leg

Chair

SLIDE 9

Generating 3D Shapes with Geometric Variations

9

Back

Back Support Back Surface Back Single Surface

Arm Arm

Sofa Arm Sofa Arm

Seat

Seat Support Seat Frame Frame Bar Frame Bar Frame Bar Frame Bar Seat Surface Seat Single Surface

Base

Regular Leg Base Leg Leg Leg Leg

Chair

SLIDE 10

10

Network Architecture: conditional-GAN

struct score

vertical bar

horizontal bar

leg1 leg2

stretcher2

...

arm

base chair

Part Tree Encoding Part Tree Feature Decoding

Part PC Decoding Part PC Encoding Tree Feature Encoding

Part-tree Conditioned Generator Part-tree Conditioned Discriminator

...

SLIDE 11

Network Architecture: conditional-GAN

vertical bar

horizontal bar

leg1 leg2

stretcher2

...

arm

base

Part Tree Encoding Part-tree Conditioned Generator

...

11

chair

SLIDE 12

Network Architecture: conditional-GAN

vertical bar

horizontal bar

leg1 leg2

stretcher2

...

arm

base chair

Part Tree Encoding Part-tree Conditioned Generator

...

Part PC Decoding

Part-tree Conditioned Generator

...

SLIDE 18

18

Network Architecture: conditional-GAN

Part PC Encoding

Part-tree Conditioned Discriminator

SLIDE 19

19

Network Architecture: conditional-GAN

Part PC Encoding Tree Feature Encoding

Part-tree Conditioned Discriminator

SLIDE 20

20

Network Architecture: conditional-GAN

struct score

Part PC Encoding Tree Feature Encoding

Part-tree Conditioned Discriminator

SLIDE 21

21

Network Architecture: conditional-GAN

struct score

Part PC Encoding Tree Feature Encoding

Part-tree Conditioned Discriminator

PointNet

whole score

SLIDE 22

22

Network Architecture: conditional-GAN

struct score

vertical bar

horizontal bar

leg1 leg2

stretcher2

...

arm

base chair

Part Tree Encoding Part Tree Feature Decoding

Part PC Decoding Part PC Encoding Tree Feature Encoding

Part-tree Conditioned Generator Part-tree Conditioned Discriminator

...

Trained End-to-end with WGAN-GP

SLIDE 23

23

Results: Part-Tree To Point-Cloud Generation

chair

back arm base

vertical bar

x 2

horizontal bar surface single surface support regular leg base bar stretcher x 4 leg

x 4 seat

surface single surface frame bar x 4

table

tabletop base

drawer base leg

x 4

surface shelf

x 2

cabinet

frame

bottom panel top panel side panel

x 2

divider panel

x 4 shelf x 2 ceiling lamp lamp base

light bulb

lamp

pendant lamp

chain x 3 holistic base base part

lamp unit

lamp head lamp cover lamp shade

SLIDE 24

24

Baseline Methods

SLIDE 25

25

Evaluation Metrics

Achlioptas et al., Learning Representations and Generative Models for 3D Point Clouds, ICML 2018 Mo et al., Structurenet: Hierarchical Graph Networks for 3D Shape Generation, Siggraph Asia 2019 Shu et al., 3D Point Cloud Generative Adversarial Network based on Tree Structured Graph Convolutions, ICCV 2019

SLIDE 26

26

HierInsSeg: Measuring Structure Reconstruction

Pre-define a Part Hierarchy Template

Mo et al., PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding, CVPR 2019

SLIDE 27

27

HierInsSeg: Measuring Structure Reconstruction

SLIDE 28

28

Results: Numbers

SLIDE 29

29

Results: Chair and Table

SLIDE 30

30

Results: Cabinet and Lamp

SLIDE 31

31

Results: User Study

SLIDE 32

32

Results: Disentangling Structure and Geometry

SLIDE 33

Thank you!

Looking forward to E-meeting you for questions!

Project: https://cs.stanford.edu/~kaichun/pt2pc Github: https://github.com/daerduoCarey/pt2pc

33