CS 4803 / 7643: Deep Learning Topics: Structured representations - - PowerPoint PPT Presentation

▶
cs 4803 7643 deep learning
SMART_READER_LITE
LIVE PREVIEW

CS 4803 / 7643: Deep Learning Topics: Structured representations - - PowerPoint PPT Presentation

Mar 27, 2024 175 likes •698 views

CS 4803 / 7643: Deep Learning Topics: Structured representations with graph networks Zsolt Kira Georgia Tech Deep Learning (C) Dhruv Batra & Zsolt Kira 2 Slide Credit: Thomas Kipf (C) Dhruv Batra & Zsolt Kira 3 Slide Credit:

slide-1
SLIDE 1

CS 4803 / 7643: Deep Learning

Zsolt Kira Georgia Tech

Topics:

– Structured representations with graph networks

slide-2
SLIDE 2

Deep Learning

(C) Dhruv Batra & Zsolt Kira 2

Slide Credit: Thomas Kipf

slide-3
SLIDE 3

(C) Dhruv Batra & Zsolt Kira 3

Slide Credit: Thomas Kipf

slide-4
SLIDE 4

(C) Dhruv Batra & Zsolt Kira 4

Slide Credit: Thomas Kipf

slide-5
SLIDE 5

(C) Dhruv Batra & Zsolt Kira 5

Slide Credit: Thomas Kipf

slide-6
SLIDE 6

(C) Dhruv Batra & Zsolt Kira 6

Slide Credit: Thomas Kipf

slide-7
SLIDE 7

(C) Dhruv Batra & Zsolt Kira 7

Slide Credit: Thomas Kipf

slide-8
SLIDE 8

(C) Dhruv Batra & Zsolt Kira 8

Slide Credit: Thomas Kipf

slide-9
SLIDE 9

(C) Dhruv Batra & Zsolt Kira 9

Slide Credit: Thomas Kipf

slide-10
SLIDE 10

(C) Dhruv Batra & Zsolt Kira 10

Slide Credit: Thomas Kipf

slide-11
SLIDE 11

(C) Dhruv Batra & Zsolt Kira 11

Slide Credit: Thomas Kipf

slide-12
SLIDE 12

(C) Dhruv Batra & Zsolt Kira 12

Slide Credit: Thomas Kipf

slide-13
SLIDE 13

(C) Dhruv Batra & Zsolt Kira 13

Slide Credit: Thomas Kipf

slide-14
SLIDE 14

(C) Dhruv Batra & Zsolt Kira 14

Slide Credit: Thomas Kipf

slide-15
SLIDE 15

(C) Dhruv Batra & Zsolt Kira 15

Slide Credit: Thomas Kipf

slide-16
SLIDE 16

(C) Dhruv Batra & Zsolt Kira 16

Slide Credit: Thomas Kipf

slide-17
SLIDE 17

(C) Dhruv Batra & Zsolt Kira 17

Slide Credit: Thomas Kipf

slide-18
SLIDE 18

(C) Dhruv Batra & Zsolt Kira 18

Slide Credit: Thomas Kipf

slide-19
SLIDE 19

(C) Dhruv Batra & Zsolt Kira 19

Slide Credit: Thomas Kipf

slide-20
SLIDE 20

(C) Dhruv Batra & Zsolt Kira 20

Slide Credit: Thomas Kipf

slide-21
SLIDE 21

(C) Dhruv Batra & Zsolt Kira 21

Slide Credit: Thomas Kipf

slide-22
SLIDE 22

(C) Dhruv Batra & Zsolt Kira 22

Slide Credit: Thomas Kipf

slide-23
SLIDE 23

(C) Dhruv Batra & Zsolt Kira 23

Slide Credit: Thomas Kipf

slide-24
SLIDE 24

(C) Dhruv Batra & Zsolt Kira 24

Slide Credit: Thomas Kipf

slide-25
SLIDE 25

(C) Dhruv Batra & Zsolt Kira 25

Slide Credit: Thomas Kipf

slide-26
SLIDE 26

(C) Dhruv Batra & Zsolt Kira 26

Slide Credit: Thomas Kipf

slide-27
SLIDE 27

(C) Dhruv Batra & Zsolt Kira 27

Slide Credit: Thomas Kipf

slide-28
SLIDE 28

(C) Dhruv Batra & Zsolt Kira 28

Slide Credit: Thomas Kipf

slide-29
SLIDE 29

(C) Dhruv Batra & Zsolt Kira 29

Slide Credit: Thomas Kipf

slide-30
SLIDE 30

(C) Dhruv Batra & Zsolt Kira 30

Slide Credit: Thomas Kipf

slide-31
SLIDE 31

(C) Dhruv Batra & Zsolt Kira 31

Slide Credit: Thomas Kipf

slide-32
SLIDE 32

(C) Dhruv Batra & Zsolt Kira 32

Slide Credit: Thomas Kipf

slide-33
SLIDE 33

(C) Dhruv Batra & Zsolt Kira 33

Slide Credit: Thomas Kipf

slide-34
SLIDE 34

(C) Dhruv Batra & Zsolt Kira 34

Slide Credit: Thomas Kipf

slide-35
SLIDE 35

(C) Dhruv Batra & Zsolt Kira 35

Slide Credit: Thomas Kipf

slide-36
SLIDE 36

(C) Dhruv Batra & Zsolt Kira 36

Slide Credit: Thomas Kipf

slide-37
SLIDE 37

(C) Dhruv Batra & Zsolt Kira 37

Slide Credit: Thomas Kipf

slide-38
SLIDE 38

(C) Dhruv Batra & Zsolt Kira 38

Slide Credit: Thomas Kipf

slide-39
SLIDE 39

(C) Dhruv Batra & Zsolt Kira 39

Slide Credit: Thomas Kipf

slide-40
SLIDE 40

Problem Description

(C) Dhruv Batra & Zsolt Kira 40

  • Application: Component detection on PCBs.
  • Input: high resolution (eg. 6000 x

4000) PCB images.

  • Output: component types

(eg. resistor, capacitor, IC, etc) and bounding boxes.

  • Challenge 1: expensive to build a large-

scale dataset for training

  • ~500 components per PCB.
  • Require professional labeler.
  • Very few high-quality datasets

available on the Internet.

Data-Efficient Graph Embedding Learning for PCB Component Detection Chia-Wen Kuo, Jacob Ashmore, David Huggins, Zsolt Kira

slide-41
SLIDE 41

Problem Description

(C) Dhruv Batra & Zsolt Kira 41

  • Challenge 2: data distribution
  • Unbalanced data distribution: 100+

resistors and capacitors, 10+ ICs, and only a few switches.

  • High intra-class variance (eg.

connector).

  • Low inter-class variance (eg.

resistor, led, capacitor).

slide-42
SLIDE 42

Graph Network (GN)

(C) Dhruv Batra & Zsolt Kira 42

  • Capture the spatial feature of

component layout.

  • Capture the structure of feature

manifold.

  • Capture the global statistics of the

whole board.

  • => Refine the feature of object

proposals based on these additional sources of information.

  • => Everything is learned and jointly
  • ptimized including graph edges and

node features.

slide-43
SLIDE 43

Similarity Prediction Network

(C) Dhruv Batra & Zsolt Kira 43

slide-44
SLIDE 44

Pipeline

(C) Dhruv Batra & Zsolt Kira 44

slide-45
SLIDE 45

Results

(C) Dhruv Batra & Zsolt Kira 45

  • Significant improvement in mAP with graph network (GN) within a board

– Triplet loss used to train similarity prediction. – Propagation of label features in few labeled examples further improves results.

slide-46
SLIDE 46

Results

(C) Dhruv Batra & Zsolt Kira 46

Blue: correct type and precise bounding box location. Cyan: imprecise bounding box location. Magenta: miss-detected component. Yellow: precise bounding box location but wrong type.

  • Leveraging the local, spatial, and global structure of

PCB boards results in significant improvements over standard detection pipelines.

  • Connectivity can be initialized via learned similarity and

jointly optimized to learn the structure to maximize accuracy.

slide-47
SLIDE 47

(C) Dhruv Batra & Zsolt Kira 47

Slide Credit: Thomas Kipf

slide-48
SLIDE 48

(C) Dhruv Batra & Zsolt Kira 48

Slide Credit: Thomas Kipf

slide-49
SLIDE 49

(C) Dhruv Batra & Zsolt Kira 49

Slide Credit: Thomas Kipf

slide-50
SLIDE 50

(C) Dhruv Batra & Zsolt Kira 50

Slide Credit: Thomas Kipf

slide-51
SLIDE 51

(C) Dhruv Batra & Zsolt Kira 51

Slide Credit: Thomas Kipf

slide-52
SLIDE 52

(C) Dhruv Batra & Zsolt Kira 52

Slide Credit: Thomas Kipf