DEEP LEARNING (F OR R OBOTIC V ISION ) Juxi Leitner @Juxi - PowerPoint PPT Presentation

H ANDS - ON T UTORIAL DEEP LEARNING (F OR R OBOTIC V ISION ) Juxi Leitner @Juxi http://Juxi.net R ESEARCHER R OBOTICS /AI h7p:/ /Juxi.net/workshop/SoAIR/

roboticvision.org

http://Juxi.net/aboutme create agents that see & interact with the real-world Pixar (2008) http://roboticvision.org/

deep learning for vision-based manipulation

H ANDS - ON T UTORIAL introduction to deep learning first contact with pytorch create a neural network train a classifier $$$ @Juxi

what is deep learning @Juxi

frameworks for deep learning @Juxi Read more at Wikipedia or check this blog for an overview https://jameskle.com/writes/deep-learning-frameworks

pytorch (2016) FAIR open source library tensor computation on GPU + autodif @Juxi

let’s play > jupyter notebook Install via conda conda create --name rvss2019 python=3 
 conda activate rvss2019 conda install jupyter conda install scipy numpy conda install pytorch=0.4.1 torchvision -c pytorch conda install opencv @Juxi https://realpython.com/jupyter-notebook-introduction/

what is deep learning @Juxi

what is machine learning supervised learning … @Juxi

what is machine learning unsupervised learning @Juxi

what is machine learning reinforcement learning run see agent act hug reward agent ? age @Juxi

machine learning classification regression @Juxi

@Juxi

@Juxi http://www.asimovinstitute.org/neural-network-zoo/

artificial neuron @Juxi

activation functions @Juxi

back propagation @Juxi

training step @Juxi

gradient descent @Juxi

train a simple neural network @Juxi

convolutional neural networks Convolution of 5 x 5 pixel image with 3 x 3 pixel filter (stride = 1 x 1 pixel) max_pooling @Juxi

convolutional neural networks https://towardsdatascience.com/image-classification-in-10-minutes-with-mnist-dataset-54c35b77a38d @Juxi

train a convolutional neural network Want to try more: 
 https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html @Juxi

break a neural network @Juxi

debugging networks http://cs231n.github.io/understanding-cnn/ use network architectures https://modelzoo.co/framework/pytorch @Juxi

deep learning visual control fully conn. + ReLU fully conn. + ReLU 7 × 7 conv + ReLU 4 × 4 conv + ReLU 3 × 3 conv + ReLU 84 × 84 fully conn. fully conn. stride 2 stride 2 stride 1 Q-values 400 units 300 units 5 units 9 units Or 64 � lters 64 � lters 64 � lters θ Conv1 Conv2 Conv3 I BN FC_c1 FC_c2 FC_c3 [Zhang et al, arxiv ] Bottleneck Control Module Perception Module Occlusion Occlusion Occlusion Occlusion A B C D E

deep learning visual control [Zhuang et al, RA:L 2018 (submitted) ]

generative grasp network [Morrison et al, RSS 2018]

thank you Juxi Leitner arc centre of excellence for robotic vision queensland university of technology j.leitner@qut.edu.au 
 http://Juxi.net Juxi