3d shapenets a deep representation for volumetric shape
play

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling - PowerPoint PPT Presentation

Jul 12, 2023 •186 likes •1.32k views

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling by Wu, Song, Khosla, Yu, Zhang, Tang, Xiao presented by Abhishek Sinha 1 3D Shape Prior 2 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric

  1. Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  2. Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  3. Dataset 21

  4. Big 3D Data Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  5. Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  6. Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category 23 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  7. Big 3D Data 151,128 models 660 categories Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  8. Applications 25

  9. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  10. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  11. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  12. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  13. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  14. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  15. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  16. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  17. 2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al

  18. 2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al

  19. 2.5D Completion & Recognition Training on CAD models and no discriminative tuning! [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. 27 Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  20. View Planning for Recognition 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  21. View Planning for Recognition Volumetric representation 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  22. View Planning for Recognition sofa? ? dresser? Volumetric representation What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  23. View Planning for Recognition sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  24. View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  25. View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  26. View Planning for Recognition Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  27. View Planning for Recognition 3D ShapeNets Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  28. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  29. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  30. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  31. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  32. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  33. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  34. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  35. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  36. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 Mathematically, this is equivalent to evaluate the conditional mutual information: 29 Slide Credit: Wu et al

  37. Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  38. Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. Our algorithm stands out as the uncertainty of the first view increases 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  39. Back Propagation Fine-tuning 3D ShapeNets 4000 object label 1200 2 512 filters of stride 1 5 160 filters of stride 2 13 48 filters of stride 2 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  40. Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  41. Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  42. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  43. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  44. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  45. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  46. Back Propagation Fine-tuning 3D ShapeNets 3D CNN 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  47. As a 3D Feature Extractor 32 Slide Credit: Wu et al

  48. As a 3D Feature Extractor Mesh Classification & Retrieval 32 Slide Credit: Wu et al

  49. As a 3D Feature Extractor Mesh Classification & Retrieval 2.5D object recognition 32 [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu et al

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Shape Representation Jin Xie Department of Computer Science and Engineering Nanjing University

Shape Representation Jin Xie Department of Computer Science and Engineering Nanjing University

Deep Learning Based 3D Shape Representation Jin Xie Department of Computer Science and Engineering Nanjing University of Science and Technology, China 1 Outline Overview of 3D deep learning Deep learned 3D shape feature for retrieval

875 views • 32 slides
Systematical Parameterization, Storage and Representation of Volumetric DICOM Data for

Systematical Parameterization, Storage and Representation of Volumetric DICOM Data for

Systematical Parameterization, Storage and Representation of Volumetric DICOM Data for Visualization: 3D Presentation States (3DPR) Dr. M. Alper SELVER Dokuz Eylul University Electrical and Electronics Engineering Research Group FH-Juelich

471 views • 19 slides
Practical Deep Learning micha.codes / fastforwardlabs.com 1 / 70 deep learning can seem

Practical Deep Learning micha.codes / fastforwardlabs.com 1 / 70 deep learning can seem

Practical Deep Learning micha.codes / fastforwardlabs.com 1 / 70 deep learning can seem mysterious 2 / 70 let's nd a way to just build a function 3 / 70 Feed Forward Layer # X.shape == (512,) # output.shape == (4,) # weights.shape ==

1.05k views • 70 slides
3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates: multi-view images depth map volumetric polygonal mesh point cloud primitive-based CAD models 3D representation Candidates: multi-view images

1.32k views • 86 slides
3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates: multi-view images depth map volumetric polygonal mesh point cloud primitive-based CAD models 3D representation Candidates: multi-view images

1.34k views • 75 slides
LAPLACE-BELTRAMI EIGENFUNCTIONS FOR DEFORMATION INVARIANT SHAPE REPRESENTATION Raif Rustamov

LAPLACE-BELTRAMI EIGENFUNCTIONS FOR DEFORMATION INVARIANT SHAPE REPRESENTATION Raif Rustamov

LAPLACE-BELTRAMI EIGENFUNCTIONS FOR DEFORMATION INVARIANT SHAPE REPRESENTATION Raif Rustamov Department of Mathematics Purdue University, West Lafayette, IN Motivation Deformable shapes Computer graphics Shape modeling

326 views • 22 slides
Chapter 8 Shape representation and description 8.1 Matching 2 8.1 Matching we wish to

Chapter 8 Shape representation and description 8.1 Matching 2 8.1 Matching we wish to

Chapter 8 Shape representation and description 8.1 Matching 2 8.1 Matching we wish to match some model to the data At simplest, known binary patterns representing characters of a font may be sought in properly aligned scans of

406 views • 27 slides
Lecture 5: Data Representation 1 / 43 Data Representation Discussion Deep learning job postings

Lecture 5: Data Representation 1 / 43 Data Representation Discussion Deep learning job postings

Data Representation Lecture 5: Data Representation 1 / 43 Data Representation Discussion Deep learning job postings have collapsed in past 6 months Something Ive learned: when non-engineers ask for an AI or ML implementation, they almost

618 views • 43 slides
Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, Spatially- - Localized Data with

Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, Spatially- - Localized Data with

Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, Spatially- - Localized Data with Localized Data with V l V l Volumetric Data Volumetric Data i D i D Robert Weersink, Harsimran Braisch, Greg Bootsma David Jaffray Greg

308 views • 20 slides
SPARSE VOLUMETRIC REPRESENTATION OF TIME-LAPSE POINT CLOUD Innfarn Yoo, 05.08.2017 Introduction

SPARSE VOLUMETRIC REPRESENTATION OF TIME-LAPSE POINT CLOUD Innfarn Yoo, 05.08.2017 Introduction

SPARSE VOLUMETRIC REPRESENTATION OF TIME-LAPSE POINT CLOUD Innfarn Yoo, 05.08.2017 Introduction Previous Work AGENDA Method Result Future Work 2 INTRODUCTION Time-lapse Point Cloud Dataset Captured external point cloud data by captured

667 views • 35 slides
Shape representation as distribution of geometric features: from currents to oriented varifolds

Shape representation as distribution of geometric features: from currents to oriented varifolds

Workshop on Statistical modeling for shapes and imaging Shape representation as distribution of geometric features: from currents to oriented varifolds Irne Kaltenmark Inria - Institut de Mathmatiques de Bordeaux Universit de Bordeaux

644 views • 47 slides
Shape Representation and Description Alexandre Falc ao Institute of Computing - University of

Shape Representation and Description Alexandre Falc ao Institute of Computing - University of

Shape Representation and Description Alexandre Falc ao Institute of Computing - University of Campinas afalcao@ic.unicamp.br Alexandre Falc ao MC920/MO443 - Indrodu c ao ao Proc. de Imagens Introduction A segmented object may

1.23k views • 77 slides
animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation many ways to represent changes with time intent artistic motion physically-plausible motion efficiency typically not a major problem control most

756 views • 74 slides
Meaning Representation and Semantic Analysis Ling 571 Deep Processing Techniques for NLP

Meaning Representation and Semantic Analysis Ling 571 Deep Processing Techniques for NLP

Meaning Representation and Semantic Analysis Ling 571 Deep Processing Techniques for NLP February 9, 2011 Roadmap Meaning representation: Event representations Temporal representation Semantic Analysis

626 views • 52 slides
Caffeine: Towards Uniformed Representation Introduction and Acceleration for Deep Convolutional

Caffeine: Towards Uniformed Representation Introduction and Acceleration for Deep Convolutional

FPGA C.Zhang et al. Caffeine: Towards Uniformed Representation Introduction and Acceleration for Deep Convolutional Motivation Neural Networks Uniformed CNN Representation Caffeine Design Chen Zhang, Zhenman Fang, Peipei Zhou et al.

445 views • 31 slides
Laplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation Author: Raif M.

Laplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation Author: Raif M.

Laplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation Author: Raif M. Rustamov Presenter: Dan Abretske Johns Hopkins 2007 Outline Motivation and Background Laplace-Beltrami Operator Global Point Signature

314 views • 18 slides
2/5/2019 Mixture Design Part 2: Maintaining the Balance Shane Buchanan CRH Americas Materials

2/5/2019 Mixture Design Part 2: Maintaining the Balance Shane Buchanan CRH Americas Materials

2/5/2019 Mixture Design Part 2: Maintaining the Balance Shane Buchanan CRH Americas Materials Learning Objectives 1. Review the fundamentals of asphalt mixtures and know what drives performance and economics. Understand the importance of

868 views • 39 slides
BERKELEY MODULAR 27 th September 2018 2018 ANNUAL REPORT How? Project teams - review

BERKELEY MODULAR 27 th September 2018 2018 ANNUAL REPORT How? Project teams - review

BERKELEY MODULAR 27 th September 2018 2018 ANNUAL REPORT How? Project teams - review off-site opportunities on each site and capture off-site spend Group - develop new approach for every site to follow Deliver the Berkeley Modular

216 views • 9 slides
3D Vision: Multi-View Stereo & Volumetric Modeling Torsten Sattler, Martin Oswald Spring

3D Vision: Multi-View Stereo & Volumetric Modeling Torsten Sattler, Martin Oswald Spring

3D Vision: Multi-View Stereo & Volumetric Modeling Torsten Sattler, Martin Oswald Spring 2018 http://www.cvg.ethz.ch/teaching/3dvision/ Schedule Feb 19 Introduction Feb 26 Geometry, Camera Model, Calibration Mar 5 Features, Tracking

698 views • 55 slides
Volumetric Modeling Schedule (tentative) Feb 18 Introduction Feb 25 Lecture: Geometry, Camera

Volumetric Modeling Schedule (tentative) Feb 18 Introduction Feb 25 Lecture: Geometry, Camera

Volumetric Modeling Schedule (tentative) Feb 18 Introduction Feb 25 Lecture: Geometry, Camera Model, Calibration Mar 4 Lecture: Features, Tracking/Matching Project Proposals by Students Mar 11 Mar 18 Lecture: Epipolar Geometry Mar 25

589 views • 35 slides
The Role of Smart Lighting in the Smart Grid St Stewart Findlater t Fi dl t VP, Engineering

The Role of Smart Lighting in the Smart Grid St Stewart Findlater t Fi dl t VP, Engineering

The Role of Smart Lighting in the Smart Grid St Stewart Findlater t Fi dl t VP, Engineering Discussing Today The smart grid needs a smart load The smart grid needs a smart load Lighting in commercial buildings LEDs a flexible

479 views • 10 slides
Slide 1 / 43 1 After firing a cannon ball a cannon moves in opposite direction from the ball.

Slide 1 / 43 1 After firing a cannon ball a cannon moves in opposite direction from the ball.

Slide 1 / 43 1 After firing a cannon ball a cannon moves in opposite direction from the ball. This an example of Newtons First law A Newtons Second Law B Newtons Third Law C Newtons Law of Gravitation D None of the above E

432 views • 15 slides
Slide 1 / 125 Slide 2 / 125 1 A block is pushed 22.0 m along a frictionless 2 A person pushes

Slide 1 / 125 Slide 2 / 125 1 A block is pushed 22.0 m along a frictionless 2 A person pushes

Slide 1 / 125 Slide 2 / 125 1 A block is pushed 22.0 m along a frictionless 2 A person pushes a stroller a distance of 9.5 m horizontal surface by a force of 23.1 N. How while performing 350 J of work on the stroller. much work does the

482 views • 21 slides
Not Quite There Yet: The Quest for the Right Environment Player Model in Games for Reactive

Not Quite There Yet: The Quest for the Right Environment Player Model in Games for Reactive

Not Quite There Yet: The Quest for the Right Environment Player Model in Games for Reactive Synthesis R udiger Ehlers University of Bremen Dagstuhl Seminar 17111, March 2017 Using joint preliminary work done together with Roderick Bloem,

757 views • 34 slides

More recommend