LegoNet: Efficient Convolutional Neural Networks with Lego Filters - - PowerPoint PPT Presentation

LegoNet: Efficient Convolutional Neural Networks with Lego Filters

1Laboratory of Machine Percep3on (Ministry of Educa3on), Peking University 2Huawei Noah’s Ark Lab 3Peng Cheng Laboratory 4Na3onal Engineering Laboratory For Video Technology, Peking University 5School of Computer Science, University of Sydney

*This work was down when Zhaohui Yang and Han3ng Chen were interns at Huawei Noah’s Ark Lab

Zhaohui Yang1,2,* Yunhe Wang2 Han3ng Chen1,2,* Chuanjian Liu2 Boxin Shi3,4 Chao Xu1 Chunjing Xu2 Chang Xu5

Goal

• Mo&va&on

Reuse paXerns

• Targeted

Build efficient CNN using a set of Lego Filters

• Lego Filters

Standard convolu3on filters are established by a set of shared filters

• Op&miza&on

End-to-end op3miza3on, Straight Through Es3mator

• Efficient Inference

Split-Transform-Merge strategy

ICML 2019

Lego Filters

• Lego Filters B

B = {B1, …, Bm}

• Standard convolu&on filters F

F = G(B1, …, Bm), 4-D tensor G is a genera3on func3on.

• Compression condi&on

|G| + |B| ≤ |F|

• G in LegoNet

Combina3on

ICML 2019

m: the number of Lego Filters B: lego Filters F: standard convolu3on filters G: genera3on func3on

Op&miza&on

ICML 2019

• Targeted
• Op&mize Lego Filters B

Standard BP algorithm

• Op&mize Binary matrix M

Float type proxy weight N Straight Through Es3mator (STE)

m: the number of Lego Filters B: Lego Filters M: binary index matrix N: proxy matrix of M

• Split
• Split input feature maps X
• Transform
• Convolve feature fragments X =

{X1, …, Xo} with Lego Filters B = {B1, …, Bm}

• Merge
• Combine Lego Feature Maps

according to learnt combina3on matrix M

Lego Unit & Efficient Inference

ICML 2019

m: the number of Lego Filters

• : split number

B: Lego Filters M: binary index matrix

• Compression
• Accelera&on
• Condi3on
• m ≤ n

Analysis

ICML 2019

m: the number of Lego Filters n: the number of output channels

• : split number

M: binary index matrix

Experiments

ICML 2019

Combina3on with coefficients is important while stacking Lego Filters. Given same model size, larger split number o results in higher performance (larger FLOPs) CIFAR-10 ImageNet

Conclusion & Future Research

ICML 2019

• Conclusion
• Proposed Lego Filters for construc3ng efficient CNN.
• End-to-end op3miza3on.
• Split-transform-merge three-stage strategy.
• Future Research
• Parameter in Parameter (use a set of Lego Filters and a small NN to generate

4-D convolu3on filters)

• Global LegoNet (view network parameters as a whole 4-D tensor)

Thanks!

ICML 2019