Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving - - PowerPoint PPT Presentation

▶
re thinking cnn frameworks for time sensitive autonomous
SMART_READER_LITE
LIVE PREVIEW

Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving - - PowerPoint PPT Presentation

Jul 01, 2023 192 likes •1.04k views

Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge Ming Yang 1 , Shige Wang 2 , Joshua Bakita 1 , Thanh Vu 1 , F. Donelson Smith 1 , James H. Anderson 1 , and Jan-Michael Frahm 1 1

slide-1
SLIDE 1

Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge

Ming Yang1, Shige Wang2, Joshua Bakita1, Thanh Vu1, F. Donelson Smith1, James H. Anderson1, and Jan-Michael Frahm1

1The University of North Carolina at Chapel Hill 2General Motors Research

slide-2
SLIDE 2

Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge

Ming Yang1, Shige Wang2, Joshua Bakita1, Thanh Vu1, F. Donelson Smith1, James H. Anderson1, and Jan-Michael Frahm1

1The University of North Carolina at Chapel Hill 2General Motors Research

slide-3
SLIDE 3

Ming Yang - RTAS 2019 3

slide-4
SLIDE 4

Ming Yang - RTAS 2019 3

slide-5
SLIDE 5

Ming Yang - RTAS 2019 3

slide-6
SLIDE 6

Ming Yang - RTAS 2019 3

slide-7
SLIDE 7

Ming Yang - RTAS 2019 3

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

slide-8
SLIDE 8

Ming Yang - RTAS 2019 3

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

Icons made by Freepik from Flaticon is licensed by CC 3.0 BY

slide-9
SLIDE 9

Ming Yang - RTAS 2019 3

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

  • 1. Response time

Icons made by Freepik from Flaticon is licensed by CC 3.0 BY

slide-10
SLIDE 10

Ming Yang - RTAS 2019 3

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

  • 1. Response time
  • 2. Accuracy

Icons made by Freepik from Flaticon is licensed by CC 3.0 BY

slide-11
SLIDE 11

Ming Yang - RTAS 2019 3

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

  • 1. Response time
  • 2. Accuracy
  • 3. Throughput

Icons made by Freepik from Flaticon is licensed by CC 3.0 BY

slide-12
SLIDE 12

Ming Yang - RTAS 2019

  • 1. Response time
  • 2. Accuracy
  • 3. Throughput

4

Our focus

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

slide-13
SLIDE 13

Ming Yang - RTAS 2019

Hardware resources are constrained and expensive.

  • 1. Response time
  • 2. Accuracy
  • 3. Throughput

4

Our focus

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

slide-14
SLIDE 14

Ming Yang - RTAS 2019

Hardware resources are constrained and expensive. CNN software underutilizes the hardware.

  • 1. Response time
  • 2. Accuracy
  • 3. Throughput

4

Our focus

https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/

slide-15
SLIDE 15

Ming Yang - RTAS 2019

Current CNN frameworks

5

CPU GPU

slide-16
SLIDE 16

Ming Yang - RTAS 2019

Current CNN frameworks

5

CPU GPU

slide-17
SLIDE 17

Ming Yang - RTAS 2019

Current CNN frameworks

5

CPU GPU

slide-18
SLIDE 18

Ming Yang - RTAS 2019

Current CNN frameworks

5

CPU GPU

Gaps

slide-19
SLIDE 19

Ming Yang - RTAS 2019

Current CNN frameworks

5

CPU GPU

Gaps Cycles not utilized

slide-20
SLIDE 20

Ming Yang - RTAS 2019

Current CNN frameworks

5

CPU GPU

Gaps Cycles not utilized

Single CNN underutilizes the hardware.

slide-21
SLIDE 21

Ming Yang - RTAS 2019 6

Private CNN

C-1

Private CNN

… … …

Traditional Multiple-Camera Processing Setup

slide-22
SLIDE 22

Ming Yang - RTAS 2019 6

Issues:

Private CNN

C-1

Private CNN

… … …

Traditional Multiple-Camera Processing Setup

slide-23
SLIDE 23

Ming Yang - RTAS 2019 6

Issues:

  • 1. Memory requirements multiply, limiting the number of instances.

Private CNN

C-1

Private CNN

… … …

Traditional Multiple-Camera Processing Setup

slide-24
SLIDE 24

Ming Yang - RTAS 2019 6

Issues:

  • 1. Memory requirements multiply, limiting the number of instances.
  • 2. Context switches on GPU cause overheads.

Private CNN

C-1

Private CNN

… … …

Traditional Multiple-Camera Processing Setup

slide-25
SLIDE 25

Ming Yang - RTAS 2019 6

Issues:

  • 1. Memory requirements multiply, limiting the number of instances.
  • 2. Context switches on GPU cause overheads.
  • 3. Fast synchronization between cameras becomes harder.

Private CNN

C-1

Private CNN

… … …

Traditional Multiple-Camera Processing Setup

slide-26
SLIDE 26

Ming Yang - RTAS 2019 6

Issues:

  • 1. Memory requirements multiply, limiting the number of instances.
  • 2. Context switches on GPU cause overheads.
  • 3. Fast synchronization between cameras becomes harder.

Private CNN

C-1

Private CNN

… … …

Traditional Multiple-Camera Processing Setup

Parallelism through multi-process isn’t helping.

slide-27
SLIDE 27

Ming Yang - RTAS 2019 7

Part I: Part II:

Proposed Solutions

Multi-camera Composite Images to provide high throughput for multiple cameras. Parallel Execution for CNN frameworks

slide-28
SLIDE 28

Ming Yang - RTAS 2019 8

Part I: Part II:

Proposed Solutions

Multi-camera Composite Images to provide high throughput for multiple cameras. Parallel Execution for CNN frameworks

slide-29
SLIDE 29

Ming Yang - RTAS 2019 9

Part I: Part II: Parallel Execution Multi- camera Composite Images

Let’s re-think the design of CNN frameworks

Ming Yang - RTAS 2019 Layer 1 Layer n

…

slide-30
SLIDE 30

Ming Yang - RTAS 2019 9

Part I: Part II: Parallel Execution Multi- camera Composite Images

Let’s re-think the design of CNN frameworks

  • CNN models are graphs of

layers.

Ming Yang - RTAS 2019 Layer 1 Layer n

…

slide-31
SLIDE 31

Ming Yang - RTAS 2019 9

Part I: Part II: Parallel Execution Multi- camera Composite Images

Let’s re-think the design of CNN frameworks

  • CNN models are graphs of

layers.

  • Processing of images can be

independent, e.g., object detection.

Ming Yang - RTAS 2019 Layer 1 Layer n

…

slide-32
SLIDE 32

Ming Yang - RTAS 2019 9

Part I: Part II: Parallel Execution Multi- camera Composite Images

Let’s re-think the design of CNN frameworks

We enable parallel execution for CNN frameworks and shared CNN for multiple cameras.

  • CNN models are graphs of

layers.

  • Processing of images can be

independent, e.g., object detection.

Ming Yang - RTAS 2019 Layer 1 Layer n

…

slide-33
SLIDE 33

Ming Yang - RTAS 2019 10

  • Generalize concept of

layers into stages

Stage 0 Stage 1 Stage N-1

…

L a y e r 퓁 L a y e r 퓁 + 퓀

…

slide-34
SLIDE 34

Ming Yang - RTAS 2019 11

Stage 0 Stage 1 Stage N-1

… 2 3 1

Queue 0 Queue 1 Queue N-1

Frames

1 2 3

Data for Frame 3

Bookkeeping data

Data for Frame 2 Data for Frame 1

1 2 3

  • Generalize concept of

layers into stages

  • Communicate data

between stages using PGMRT (a processing graph management tool)

slide-35
SLIDE 35

Ming Yang - RTAS 2019 12

Stage 0 Stage 1 Stage N-1

… 2 3 1

Queue 0 Queue 1 Queue N-1

C-1

…

Cameras

Shared CNN

Detection box results

…

  • Generalize concept of

layers into stages

  • Communicate data

between stages using PGMRT (a processing graph management tool)

  • Share CNN among

multiple cameras

Frames

1 2 3

Data for Frame 3

Bookkeeping data

Data for Frame 2 Data for Frame 1

1 2 3

slide-36
SLIDE 36

Ming Yang - RTAS 2019 13

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Different Execution Methods

slide-37
SLIDE 37

Ming Yang - RTAS 2019 13

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL private CNN in one process

Different Execution Methods

slide-38
SLIDE 38

Ming Yang - RTAS 2019 13

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL private CNN in one process PIPELINE shared CNN that has one thread per stage

Different Execution Methods

slide-39
SLIDE 39

Ming Yang - RTAS 2019 13

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL private CNN in one process PIPELINE shared CNN that has one thread per stage PARALLEL shared CNN that has multiple threads per stage

Different Execution Methods

slide-40
SLIDE 40

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

slide-41
SLIDE 41

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

slide-42
SLIDE 42

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

slide-43
SLIDE 43

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

slide-44
SLIDE 44

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

slide-45
SLIDE 45

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

slide-46
SLIDE 46

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

slide-47
SLIDE 47

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

Concurrency

slide-48
SLIDE 48

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

Concurrency

slide-49
SLIDE 49

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

Concurrency

slide-50
SLIDE 50

Ming Yang - RTAS 2019 14

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

SERIAL PIPELINE PARALLEL

CPU GPU CPU GPU CPU GPU

Concurrency

slide-51
SLIDE 51

Ming Yang - RTAS 2019 15

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation

  • We compared latency and throughput between
  • SERIAL
  • SERIAL x6
  • PIPELINE
  • PARALLEL
  • With CNN model Tiny YOLOv2 on CNN framework

Darknet

  • On hardware platform: NVIDIA Drive PX 2.
slide-52
SLIDE 52

Ming Yang - RTAS 2019

NVIDIA Drive PX 2

16

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation (Hardware)

SoC Tegra A SoC Tegra B dGPU dGPU

slide-53
SLIDE 53

Ming Yang - RTAS 2019

NVIDIA Drive PX 2

17

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation (Hardware)

SoC Tegra A SoC Tegra B dGPU dGPU

slide-54
SLIDE 54

Ming Yang - RTAS 2019 18

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results

SERIAL PIPELINE PARALLEL SERIAL x6

slide-55
SLIDE 55

Ming Yang - RTAS 2019 18

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Higher frame rates (throughput) the better

Evaluation Results

SERIAL PIPELINE PARALLEL SERIAL x6

slide-56
SLIDE 56

Ming Yang - RTAS 2019 18

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Higher frame rates (throughput) the better Lower latency the better

Evaluation Results

SERIAL PIPELINE PARALLEL SERIAL x6

slide-57
SLIDE 57

Ming Yang - RTAS 2019 19

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results

SERIAL PIPELINE PARALLEL SERIAL x6

slide-58
SLIDE 58

Ming Yang - RTAS 2019 19

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results

SERIAL PIPELINE PARALLEL SERIAL x6

slide-59
SLIDE 59

Ming Yang - RTAS 2019 20

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results

SERIAL PIPELINE PARALLEL SERIAL x6

slide-60
SLIDE 60

Ming Yang - RTAS 2019 21

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results

SERIAL PIPELINE PARALLEL SERIAL x6

slide-61
SLIDE 61

Ming Yang - RTAS 2019 22

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

CPUs (%) Memory (MB)

SERIAL

92 774

SERIAL X6

536 4,644

PIPELINE

(Single thread per stage)

219 1,132

PARALLEL

(10 threads per stage)

239 1,136

Evaluation Results (cont.)

slide-62
SLIDE 62

Ming Yang - RTAS 2019 23

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results (cont.)

CPUs (%) Memory (MB)

SERIAL

92 774

SERIAL X6

536 4,644

PIPELINE

(Single thread per stage)

219 1,132

PARALLEL

(10 threads per stage)

239 1,136 4,644 536

slide-63
SLIDE 63

Ming Yang - RTAS 2019 24

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

CPUs (%) Memory (MB)

SERIAL

92 774

SERIAL X6

536 4,644

PIPELINE

(Single thread per stage)

219 1,132

PARALLEL

(10 threads per stage)

239 1,136

CPU and memory overheads are acceptable.

Evaluation Results (cont.)

slide-64
SLIDE 64

Ming Yang - RTAS 2019 25

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

CPUs (%) Memory (MB)

SERIAL

92 774

SERIAL X6

536 4,644

PIPELINE

(Single thread per stage)

219 1,132

PARALLEL

(10 threads per stage)

239 1,136

Enabling intra-stage parallelism takes slight overheads.

Evaluation Results (cont.)

slide-65
SLIDE 65

Ming Yang - RTAS 2019

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Part I: Parallel Execution

  • Pipeline and Parallel improve throughput from 28 FPS

to 71 FPS

  • With acceptable overheads and
  • No accuracy loss

26

slide-66
SLIDE 66

Ming Yang - RTAS 2019 27

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Icons made by Butterflytronics from Flaticon is licensed by CC 3.0 BY

slide-67
SLIDE 67

Ming Yang - RTAS 2019 28

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Icons made by Butterflytronics from Flaticon is licensed by CC 3.0 BY

slide-68
SLIDE 68

Ming Yang - RTAS 2019 29

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Icons made by Butterflytronics from Flaticon is licensed by CC 3.0 BY

slide-69
SLIDE 69

Ming Yang - RTAS 2019 30

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

slide-70
SLIDE 70

Ming Yang - RTAS 2019 31

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Virtual Camera

slide-71
SLIDE 71

Ming Yang - RTAS 2019 31

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Images are from PASCAL dataset Virtual Camera

slide-72
SLIDE 72

Ming Yang - RTAS 2019 31

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Images are from PASCAL dataset

Shared CNN

Virtual Camera

slide-73
SLIDE 73

Ming Yang - RTAS 2019 31

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Images are from PASCAL dataset

Shared CNN

Virtual Camera

slide-74
SLIDE 74

Ming Yang - RTAS 2019 31

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Multi-camera Composite Images

Images are from PASCAL dataset

Shared CNN

Virtual Camera

slide-75
SLIDE 75

Ming Yang - RTAS 2019 32

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

  • We compared latency, throughput and accuracy

between

  • Full-size images
  • Four-camera composite images

Evaluation

slide-76
SLIDE 76

Ming Yang - RTAS 2019 33

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results

Images are from PASCAL dataset

slide-77
SLIDE 77

Ming Yang - RTAS 2019 34

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results (cont.)

Images are from PASCAL dataset

slide-78
SLIDE 78

Ming Yang - RTAS 2019 34

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results (cont.)

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Full-size Test

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Composite Test

Images are from PASCAL dataset

Classes: bicycle, bus, car, motorbike, train, bird, person, cat, cow, dog, horse, sheep

slide-79
SLIDE 79

Ming Yang - RTAS 2019 34

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results (cont.)

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Full-size Test

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Composite Test

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Original YOLO

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

63.66

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

44.91

Images are from PASCAL dataset

Classes: bicycle, bus, car, motorbike, train, bird, person, cat, cow, dog, horse, sheep

slide-80
SLIDE 80

Ming Yang - RTAS 2019 34

Part I: Part II: Parallel Execution Multi- camera Composite Images

Ming Yang - RTAS 2019

Evaluation Results (cont.)

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Full-size Test

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Composite Test

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Original YOLO

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

63.66

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

44.91

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

Retrained YOLO

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

66.20

Table 1: accuracy (mAPs) of object classes relevant to autonomous driving

56.21

Images are from PASCAL dataset

Classes: bicycle, bus, car, motorbike, train, bird, person, cat, cow, dog, horse, sheep

slide-81
SLIDE 81

Ming Yang - RTAS 2019

Conclusions

  • We presented an industrial study that addresses

the challenge of supporting multiple cameras.

  • Parallel execution
  • Multi-camera composite image
  • Evaluation results showed
  • Significant throughput improvements
  • No accuracy loss with parallel execution
  • ~7.4% accuracy drop with multi-camera

composite image (but 4-fold throughput improvement!)

35

Table 1: mAPs of object classes relevant to autonomous driving Full-size Test Composite Test Original YOLO 63.66 44.91 Retrained YOLO 66.20 56.21

Images are from PASCAL dataset

slide-82
SLIDE 82

Ming Yang - RTAS 2019

Conclusions

  • We presented an industrial study that addresses

the challenge of supporting multiple cameras.

  • Parallel execution
  • Multi-camera composite image
  • Evaluation results showed
  • Significant throughput improvements
  • No accuracy loss with parallel execution
  • ~7.4% accuracy drop with multi-camera

composite image (but 4-fold throughput improvement!)

35

Other considerations in the paper:

  • Configurable stages
  • Multi-GPU execution

Table 1: mAPs of object classes relevant to autonomous driving Full-size Test Composite Test Original YOLO 63.66 44.91 Retrained YOLO 66.20 56.21

Images are from PASCAL dataset

slide-83
SLIDE 83

Ming Yang - RTAS 2019

Future Work

  • Dynamically apply composite-image technique with criticality

change

  • Finer granularity of stages
  • Dynamically share CNN among multiple models

36

slide-84
SLIDE 84

Re-thinking CNN Frameworks for Time-Sensitive Autonomous- Driving Applications: Addressing an Industrial Challenge

Ming Yang1, Shige Wang2, Joshua Bakita1, Thanh Vu1, F. Donelson Smith1, James H. Anderson1, and Jan-Michael Frahm1

1The University of North Carolina at Chapel Hill 2General Motors Research

Thank you!

Ming Yang - RTAS 2019

37