FROM SCRATCH TO PRODUCTION Andrew Liu, Ryan Shen Deep Learning - - PowerPoint PPT Presentation

▶
from scratch to production
SMART_READER_LITE
LIVE PREVIEW

FROM SCRATCH TO PRODUCTION Andrew Liu, Ryan Shen Deep Learning - - PowerPoint PPT Presentation

Jun 30, 2023 351 likes •695 views

DEFECT INSPECTION FROM SCRATCH TO PRODUCTION Andrew Liu, Ryan Shen Deep Learning Solution Architect Defect Inspection and its challenges NGC Docker images AGENDA Model set up - Unet Data preparation - DAGM Production - Speed up with

slide-1
SLIDE 1

Andrew Liu, Ryan Shen Deep Learning Solution Architect

DEFECT INSPECTION FROM SCRATCH TO PRODUCTION

slide-2
SLIDE 2

2

AGENDA

Defect Inspection and its challenges NGC Docker images Model set up - Unet Data preparation - DAGM Production - Speed up with TensorRT

slide-3
SLIDE 3

3

INDUSTRIAL DEFECT INSPECTION

slide-4
SLIDE 4

4

INDUSTRIAL DEFECT INSPECTION

slide-5
SLIDE 5

5

NGC DOCKER IMAGES

slide-6
SLIDE 6

Benefits for Deep Learning Workflow

High Level Benefits and Feature Set

Single software stack Develop once, deploy anywhere Scale across teams of practitioners Developer, DevOp, QC

slide-7
SLIDE 7

7

Defect inspection Workflow

from scratch to production within container Training Inference

NGC optimized docker image TENSORRT  GRE

docker pull nvcr.io/nvidia/tensorflow :18.02-py3 docker pull nvcr.io/nvidia/tensorrt: 18.02-py2

slide-8
SLIDE 8

8

MODEL SET UP

slide-9
SLIDE 9

9

WAYS FOR DEFECT INSPECTION

Depends on domain adaptation

Image Segmentation Object Detection Image Classification Label: 0 / 1 (Defect / Non Defect) Label: Bounding-Box Label: Polygons Mask

slide-10
SLIDE 10

10

CNN STRUCTURE

LeNet

Source: Design of Deep Convolutional Neural Network Architectures for Automated Feature Extraction in Industrial Inspection, D. Weimer et al, 2016

slide-11
SLIDE 11

11 “defect class1” 1000-dim vector < 1 millisecond

convnets perform classification

end-to-end learning

https://arxiv.org/abs/1605.06211

slide-12
SLIDE 12

12 ~1/10 second end-to-end learning

???

lots of pixels, little time?

https://arxiv.org/abs/1605.06211

slide-13
SLIDE 13

5122 5122 5122 1 16 16 16 32 32 32 64 64 64 128 128 256 128 256 256 2562 2562 2562 1282 1282 1282 642 642 642 322 322 322 1282 1282 1282 128 64 64 128 128 642 642 642 64 32 32 2562 2562 2562 5122 5122 5122 32 16 16

3X3 Conv2d+ReLU 2X2 MaxPool 2X2 Conv2dTranspose copy and concatenate

U-Net structure

slide-14
SLIDE 14

14

KERAS IMPLEMENTATION

Convolution

slide-15
SLIDE 15

15

DECODING

deconvolution

slide-16
SLIDE 16

16

Image segmentation on medical images

Same process among various use cases

MRI image Left ventricle heart disease Data Science BOWL 2016 Data Science BOWL 2017 CT image Nodule Lung cancer Image Nuclei Drug discovery Data Science BOWL 2018

slide-17
SLIDE 17

17

DATA PREPARATION

slide-18
SLIDE 18

18

INDUSTRIAL OPTICAL INSPECTION

German Association for Pattern Recognition

  • http://resources.mpi-inf.mpg.de/conferences/dagm/2007/prizes.html
slide-19
SLIDE 19

19

INDUSTRIAL OPTICAL INSPECTION

German Association for Pattern Recognition

Pass NG Pass NG NG Pass Pass NG

slide-20
SLIDE 20

20

DATA DETAILS

  • Original images are 512 x 512 grayscale format
  • Output is a tensor of size 512 x 512 x 1
  • Each pixel belongs to one of two classes
  • Training set consist of 100 images
  • Validation set consist of 50 images
slide-21
SLIDE 21

21

MORE EXAMPLES

slide-22
SLIDE 22

22

IMBALANCE DATA

slide-23
SLIDE 23

23

Dice Metric (IOU)

  • Metric to compare the similarity of two samples:

2𝐵𝑜𝑚

________________________________

𝐵𝑜 + 𝐵𝑚

  • Where:
  • An is the area of the contour predicted by the network
  • Al is the area of the contour from the label
  • Anl is the intersection of the two
  • The area of the contour that is predicted correctly by the network
  • 1.0 means perfect score.
  • More accurately compute how well we’re predicting the contour against the label
  • We can just count pixels to give us the respective areas
slide-24
SLIDE 24

24

LOSS FUNCTION WITH KERAS

24

slide-25
SLIDE 25

25

LEARNING CURVES

25 LOSS

slide-26
SLIDE 26

26

APPLICATION: INDUSTRIAL INSPECTION

NVIDIA

  • 1000 defect-free, 150 defect images
  • Challenges: Not all deviations from the

texture are necessarily defects.

slide-27
SLIDE 27

27

PRODUCTION

slide-28
SLIDE 28

28

FINAL DECISION

Plus Human Logic

Size, Position, … etc

slide-29
SLIDE 29

29

INFERENCE PIPELINE

Post-processing and decision making Inference

Camera Machine P40 DGX Server TensorRT+GRE Fetch image TensorRT+GRE

Detectors/Classifiers/Segment Composite Result Metadata

DataCenter Edge

 Defect Pattern Ratio  Defect Level  Defect region size  Defect counts …

Domain Criteria Determine threshold

Precision/ Recall

slide-30
SLIDE 30

30

Development flow

Called UFF, Universal Framework Format

ONNX

slide-31
SLIDE 31

31

Defect inspection Workflow

from scratch to production within container Training Inference

NGC optimized docker image TENSORRT  GRE

docker pull nvcr.io/nvidia/tensorflow :18.02-py3 docker pull nvcr.io/nvidia/tensorrt: 18.02-py3

slide-32
SLIDE 32

32

SUMMARY

Challenges Delivers

Training , inference environment is hard to build, maintain, share. Using NGC Docker images. Model optimizations and speed up throughput. TensorRT + GRE SDK So many deep learning model out there, how to choose the right model? If your dataset, demand requirement fit the scenario like we do. U-Net model is great choice for segmentation task.

slide-33
SLIDE 33

33

Thank You