GENERATION USING 3D PIPELINES Stefan Schoenefeld, GTC 2017 - - PowerPoint PPT Presentation

▶
generation using 3d pipelines
SMART_READER_LITE
LIVE PREVIEW

GENERATION USING 3D PIPELINES Stefan Schoenefeld, GTC 2017 - - PowerPoint PPT Presentation

Sep 22, 2023 164 likes •361 views

SYNTHETIC TRAINING IMAGE GENERATION USING 3D PIPELINES Stefan Schoenefeld, GTC 2017 sschoenefeld@nvidia.com TRAINING WITH GENERATED IMAGES GENERATE TRAIN DETECT 2 BUT WHY? ACCELERATE ANNOTATE SIMULATE Rendering is fast Annotation is

slide-1
SLIDE 1

Stefan Schoenefeld, GTC 2017 sschoenefeld@nvidia.com

SYNTHETIC TRAINING IMAGE GENERATION USING 3D PIPELINES

slide-2
SLIDE 2

2

TRAINING WITH GENERATED IMAGES

TRAIN GENERATE DETECT

slide-3
SLIDE 3

3

…BUT WHY?

SYNTHIA Dataset

SIMULATE ANNOTATE ACCELERATE

Environmental effects Create new scenarios Annotation is trivial Bounding boxes Image segmentation Rendering is fast Unlimited amount of combinations of cameras, lights and objects

slide-4
SLIDE 4

4

CASE STUDY IMAGE SEGMENTATION

slide-5
SLIDE 5

5

SHUFFLE

WORKFLOW

NVIDIA DIGITS 5.0 OBJECT SYNTH PLUGIN NVPRO-PIPELINE 3D OBJECT DATASET IMAGE DATASET CREATE TRAIN 3D OBJECT DATASET FCN

slide-6
SLIDE 6

6

SIMPLE 3D OBJECTS

PROOF OF CONCEPT

slide-7
SLIDE 7

7

RESULTS

slide-8
SLIDE 8

8

CASE STUDY IMAGE CATEGORISATION

slide-9
SLIDE 9

9

NVIDIA BOXES

Create 3D dataset from artwork Textures instead of shapes Render using IRAY/UE4 Train network to categorise

BOX DETECTION

slide-10
SLIDE 10

10

QUADRO GP100

NVIDIA BOXES

...they all look the same

QUADRO M6000 24GB QUADRO M4000 QUADRO P600

slide-11
SLIDE 11

11

SETUP

  • Iterative, start simple and increase

complexity

  • Raytracing on an iray cluster
  • UE4 for „traditional“ rendering
  • 1024x10124 images, ~2500 images per

category, 15 categories

  • Using a googlenet DNN with a random

crop of 768x768

slide-12
SLIDE 12

12

GOOD RESULTS

slide-13
SLIDE 13

13

BAD RESULTS

slide-14
SLIDE 14

14

REAL WORLD ISSUES

SHRINK WRAP

slide-15
SLIDE 15

15

RESULTS CTD

AFTER ADDING SHRINK WRAP TO THE RENDERING

slide-16
SLIDE 16

16

NEXT STEPS

slide-17
SLIDE 17

17

NEXT STEPS

Everything in the same framework

FULLY INTEGRATED

Combining and creating natural scenes Advanced positioning

MULTIPLE OBJECTS

Real time generation while training No more „Out-of-space“ Scene overhead

ON DEMAND

slide-18
SLIDE 18

THANK YOU AND ENJOY THE PARTY!

slide-19
SLIDE 19

19

REFERENCES

Stefan Schoenefeld sschoenefeld@nvidia.com NVIDIA Digits https://github.com/NVIDIA/DIGITS NVPro-Pipeline https://github.com/nvpro-pipeline/pipeline Synthia Dataset http://synthia-dataset.net/