Data sets Niloy Mit Ni Mitra Ias asonas Kok okkin inos os - - PowerPoint PPT Presentation

▶
data sets
SMART_READER_LITE
LIVE PREVIEW

Data sets Niloy Mit Ni Mitra Ias asonas Kok okkin inos os - - PowerPoint PPT Presentation

Nov 06, 2023 20 likes •365 views

Deep Learning for Graphics Data sets Niloy Mit Ni Mitra Ias asonas Kok okkin inos os Pau aul l Gu Guer errero Vl Vladim imir ir Ki Kim Kos ostas Rematas Tob obias Ri Ritschel UCL UCL/Facebook UCL Adobe Research U

slide-1
SLIDE 1

Ni Niloy Mit Mitra Ias asonas Kok

  • kkin

inos

  • s

Pau aul l Gu Guer errero Vl Vladim imir ir Ki Kim Kos

  • stas Rematas

Tob

  • bias Ri

Ritschel UCL UCL/Facebook UCL Adobe Research U Washington UCL

Deep Learning for Graphics

Data sets

slide-2
SLIDE 2

EG Course “Deep Learning for Graphics”

Timetable

2

Niloy Iasonas Paul Vova Kostas Tobias Introduction X X X X Theory X NN Basics X X Supervised Applications Data X Unsupervised Applications X Beyond 2D X X X Outlook X X X X X X

slide-3
SLIDE 3

EG Course “Deep Learning for Graphics”

Modalities

  • 2D images
  • 3D datasets
  • Data augmentation
  • Synthetic image training data

3

slide-4
SLIDE 4

EG Course “Deep Learning for Graphics”

2D image data sets

4

slide-5
SLIDE 5

EG Course “Deep Learning for Graphics”

Image Datasets

  • MNIST
  • Handwritten digits
  • 28x28 images
  • 10 classes
  • 60k train/10k test

5

slide-6
SLIDE 6

EG Course “Deep Learning for Graphics”

Image Datasets

  • CIFAR10
  • Object images
  • 10 classes
  • 32 x 32 pixels
  • 50k train/10k test

6

slide-7
SLIDE 7

EG Course “Deep Learning for Graphics”

Image Datasets

  • PASCAL VOC
  • Multiple objects per image
  • 20 classes
  • Labels for classification, segmentation, detection

7

slide-8
SLIDE 8

EG Course “Deep Learning for Graphics”

Image Datasets

  • ImageNet
  • The main ”fuel” for deep learning
  • 1000 classes
  • Classification/Detection (200 classes)
  • Structure from WordNet

8

slide-9
SLIDE 9

EG Course “Deep Learning for Graphics”

Image Datasets

  • MS COCO
  • Boost to DL class/instance segmentation and

keypoint detection

  • 80 classes
  • 200k images
  • Instance segmentation masks (>1 mil)
  • Human keypoints (250k)

9

slide-10
SLIDE 10

EG Course “Deep Learning for Graphics”

LDR 2 HDR

  • http://hdrv.org/hdrcnn/material/testset/index.html

10

slide-11
SLIDE 11

EG Course “Deep Learning for Graphics”

3D data sets

11

slide-12
SLIDE 12

EG Course “Deep Learning for Graphics”

3D Datasets

  • ShapeNet
  • Similar to ImageNet but for CAD

models

  • 55 common categories
  • 10k+ models
  • ShapeNetCore
  • 12k models
  • Additional annotations (real world

dimensions, materials,…)

12

slide-13
SLIDE 13

EG Course “Deep Learning for Graphics”

3D Datasets

  • SMPL
  • Parametric human shape model
  • 72 parameters control pose and

human shape

  • Fully differentiable
  • Useful for human shape estimation,

motion capture etc

13

slide-14
SLIDE 14

EG Course “Deep Learning for Graphics”

Data Augmentation

14

slide-15
SLIDE 15

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Augment existing data with image operations to reduce overfitting
  • Much larger dataset
  • Learn expected transformations

15

slide-16
SLIDE 16

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Mirror

16

slide-17
SLIDE 17

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Mirror
  • Rotation

17

slide-18
SLIDE 18

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Mirror
  • Rotation
  • Translation

18

slide-19
SLIDE 19

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Mirror
  • Rotation
  • Translation
  • Zoom

19

slide-20
SLIDE 20

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Mirror
  • Rotation
  • Translation
  • Zoom
  • Blur

20

slide-21
SLIDE 21

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Mirror
  • Rotation
  • Translation
  • Zoom
  • Blur
  • Noise

21

slide-22
SLIDE 22

EG Course “Deep Learning for Graphics”

Data Augmentation

  • Mirror
  • Rotation
  • Translation
  • Zoom
  • Blur
  • Noise
  • Color transforms

22

Library: https://github.com/codebox/image_augmentor

slide-23
SLIDE 23

EG Course “Deep Learning for Graphics”

Creating Your Own

23

slide-24
SLIDE 24

EG Course “Deep Learning for Graphics”

Material/Illumination

24

slide-25
SLIDE 25

EG Course “Deep Learning for Graphics”

Material/Illumination

25

slide-26
SLIDE 26

EG Course “Deep Learning for Graphics”

Decomposition

26

slide-27
SLIDE 27

EG Course “Deep Learning for Graphics”

Synthetic data

27

slide-28
SLIDE 28

EG Course “Deep Learning for Graphics”

Synthetic Data

28

slide-29
SLIDE 29

EG Course “Deep Learning for Graphics”

Synthetic Data for DL

  • 3D models + renderer = unlimited data
  • Suitable for data hungry approaches such as deep networks
  • Higher fidelity -> smaller discrepancy between synthetic and real

29

slide-30
SLIDE 30

EG Course “Deep Learning for Graphics”

How To Generate Synthetic Data

  • What you need
  • 3D models with task-specific annotation
  • Renderer
  • Example: Indoor depth estimation (McCormac et al)

30

3D Room with furniture NVIDIA OptiX renderer

+ =

Data with annotation

slide-31
SLIDE 31

EG Course “Deep Learning for Graphics”

How To Generate Synthetic Data

  • How much fidelity?

31

OpenGL Natural Illumination + AO Path tracer, global Illumination (VRay)

slide-32
SLIDE 32

EG Course “Deep Learning for Graphics”

Selecting viewpoint

  • Object in the center
  • Sample from the hemisphere
  • Multiple FOV, Target, Up, etc
  • Scene
  • Simulate human camera path
  • Optimize camera position for a particular
  • bjective (eg segmentation)

32

slide-33
SLIDE 33

EG Course “Deep Learning for Graphics”

Selecting Appearance

  • Materials
  • Capture the variability of real world
  • bjects
  • BRDF, textures (MERL DB)
  • Illumination
  • Capture the effect of environment
  • Increase realism
  • Laval indoor DB

http://indoor.hdrdb.com/

33

slide-34
SLIDE 34

EG Course “Deep Learning for Graphics”

Thank you!

http://geometry.cs.ucl.ac.uk/dl4g/

34