Sony's deep learning software "Neural Network Libraries/Console - PowerPoint PPT Presentation

Sony's deep learning software "Neural Network Libraries/Console“ and its use cases in Sony Yoshiyuki Kobayashi – Senior Machine Learning Researcher Sony Network Communications Inc. / Sony Corporation GTC 2018

Sony's deep learning software Neural Network Libraries Neural Network Console nnabla.org dl.sony.com Deep learning framework with Python API GUI based deep learning IDE Open source (Apache 2.0 license) Windows version is now available for free . 2

Just released in 2017 Already utilized in many products and services. Used since 2011 and more than 1,000 users in Sony group. • • 2011 ～ 2013 ～ Open Sourced 1st gen 2016 ～ 3rd gen framework 2nd gen framework Neural Network Libraries at Jul. 2017 framework Released Windows 2015 ～ GUI Tool Version at Aug. 2017 Neural Network Console for Free • Sony real estate AR Effect Xperia Ear Xperia Hello aibo 3

Need to make deep learning research & development more efficient Motivation Also, human resource development needs to be accelerated Deep Learning is great and demand is explosively growing Deep learning engineer is lacking worldwide SOLD • • 4

Introduction of Neural Network Console https://www.youtube.com/watch?v=-lXjnaUSEtM 5

Application Example #1：Sony Real Estate Real Estate Price Estimate Neural Network Libraries is used in Real Estate Price Estimate Engine of Sony Real Estate Corporation. the Library realizes the solution that statistically estimates signed price in buying and selling real estate, analyzing massive data with unique algorism developed based on evaluation know-how and knowledge of Sony Real Estate Corporation. The solution is utilized in various businesses of Sony Real Estate Corporation such as ”Ouchi Direct”, “Real Estate Property Search Map” and “Automatic Evaluation”. Input Output Price of a real estate Bunch of features about a real estate Total floor area Floor plan … Street address … 6

Application Example #2：Xperia Ear Gesture Sensitivity The Library is used in an intuitive gesture sensitivity function of Sony Mobile Communications “Xperia Ear”. Based on data from several sensors embedded in Xperia Ear, you can just use a nod of the head to confirm a command - answering Yes/No, answering/declining the phone call, cancelling text-to-speech reading of notifications, skipping/rewinding a song track. Input Output Head gestures such as Yes and No Sensor data … 7

Application Example #3：aibo Image Recognition (User Identification, Face Tracking, etc.) The Library is used to realize image recognition of Sony’s Entertainment Robot "aibo" 『 ERS-1000 』 . In the image recognition through fish-eye cameras installed at the nose, the Library is actively utilized for user identification, face tracking, charge stand recognition, generic object recognition, etc. These features and various inbuilt sensors enable its adaptable behavior. 入力 Output Image Face, object, Charge stand, etc… … 8

Application of DL spreading depending on input / output Function Input Output … Function Input Output Image recognition Image Category of image Image filtering Source image target image Speech recognition Speech String Machine translation (En->Ja) English word string Japanese word string Chat-bot Input word string String of response Sensor error detection Sensor signal Abnormality level Robot control Robot's sensor Robot actuator … The possibilities of application are infinite 9

Feature 1. Ideal for deep learning beginners  Easy to setup Just unzip the downloaded file and run the application • Select “GPU” in Setup window to use NVIDIA GPU •  You can learn deep learning visually Convolutional  Trial and error in a short time Neural Networks Multi layer perceptron Logistic regression Provides shortcut for skill improvement on deep learning 10

Feature 2. Realize very efficient development  Easy debugging  Automatic structure search Instantly locate the error on GUI search for better neural network • • structure automatically  Easy to deploy Created model can be deployed immediately by using Neural Network Libraries • NNL offers both Python and C++ API • 11

etc… More features signal processing, regression, etc.  Functions  Management of trial and error history  Visualization of weights  Export python code  Supports …  Various data as well as images, matrices and vectors  Various objectives such as classification, detection,  Multiple inputs and outputs for multi-modal application  Training of huge neural networks like ResNet-152  Recurrent neural networks (RNN)  Generative Adversarial Networks (GAN)  Semi-supervised learning  Transfer learning 12

https://github.com/sony/nnabla-examples/ Supports both static and dynamic NNs https://github.com/sony/nnabla/tree/master/tutorial Jupyter Notebook Tutorials ResNet, GANs, CapsNet, Quantized Nets, … Neural Network Libraries Resources Supports fp16 and Tensor Cores Supports distributed training NNabla-Examles Fast training and inference with CUDA/cuDNN Portable to C++ Cross-platform & Cross-device Easy to add a new NN operator Flexible for arbitrary NNs Easy-to-write & easy-to-read https://github/sony/nnabla/  Lightweight C++ core library with Python API • pip install nnabla • • import nnabla as nn •  Sophisticated import nnabla.functions as F import nnabla.parametric_functions as PF • • x = nnabla.Variable((batch_size, 1, 28, 28)) • c1 = PF.convolution(x, 16, (5, 5), name='c1') • c1 = F.relu(F.max_pooling(c1, (2, 2))) • c2 = PF.convolution(c1, 16, (5, 5), name='c2') • c2 = F.relu(F.max_pooling(c2, (2, 2))) f3 = F.relu(PF.affine(c2, 50, name='f3')) • y = PF.affine(f3, 50, name='f4') t = Variable((batch_size, 1)) • loss = F.mean(F.softmax_cross_entropy(y, t)) … Suitable for both research and production use 13

Sony's deep learning software Neural Network Libraries Neural Network Console nnabla.org dl.sony.com Deep learning framework with Python API GUI based deep learning IDE Open source (Apache 2.0 license) Windows version is now available for free . Sony is providing an integrated development environment with hopes that a wider range of developers and researchers will build on its programs, and with the aim of contributing to the development of society. 14

Appendix 16

Tutorial Image classification (hand-writing digits recognition) Input : image Classifier Output : Classification result （ Neural Network ） 2 … 28x28 pixel … … … … … Number of output neural is number Number of input neuron 784 10 of categories of target data is size of input data … Take a typical application “ Hand-writing digits recognition ” as an example 17

Work required to create a image classifier 1. Prepare dataset Prepare as much data as possible Each data includes image and its class index … 「 0 」 … 「 1 」 … 「 2 」 … 「 3 」 … 「 4 」 Input : Image Classifier Output : … 「 5 」 （ Neural Network ） Classification result … 「 6 」 … 「 7 」 … 「 8 」 … 「 9 」 2 … 2. Design neural network structure You can use sample project … … … … … … 3. Train designed neural network using prepared dataset Basically you can create image classifier with these three steps 18

1. Prepare dataset • Create dataset CSV files according to a predetermined format class Index File name of hand-writing digit x:image y:label ./training/5/0.png 5 ./training/0/1.png 0 ./training/4/2.png 4 ./training/1/3.png 1 ./training/9/4.png 9 ./training/2/5.png 2 ./training/1/6.png 1 … It can be created with Excel or simple script 19

2. Design neural network structure • Put layers as it is by drag & drop, • Convolutional Neural Networks and edit properties of layers 5 x 5 Convolution Activation 2 x 2 Pooling 3 x 3 Convolution Activation 2 x 2 Pooling Add layers by drag & Affine drop Activation Fully connected Softmax Categorical Loss Edit property of layer You can design the network structure very easily and intuitively 20

3. Train designed neural network using prepared dataset • Training starts just by pushing the training button loss training iteration When the training is complete, an image classifier is obtained 21