Deep Learning for Automated Systems: From the Warehouse to the Road - PowerPoint PPT Presentation

Deep Learning for Automated Systems: From the Warehouse to the Road Dr. Melissa C. Smith Clemson University Future Computing Technologies Laboratory Eddie Weill, Sufeng Niu, Colin Targonski, and Ben Shealy

Overview

Simulation Using deep learning in a simulated environment

The Need for a Simulator

Deep Learning in a Simulator Webpage: http://www.carla.org/ Paper: Dosovitskiy, Alexey, et al. "CARLA: An open urban driving simulator." arXiv preprint arXiv:1711.03938 (2017).

Perception Developing perception for automated systems

Autonomous Driving Perception

Autonomous Driving Perception

Autonomous Driving Perception Tetreault, Jesse. Deep Multimodal Fusion Networks for Semantic Segmentation . Diss. Clemson University, 2017.

Perception on Embedded Devices Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Synthetic Image Proceedings of the IEEE conference on computer vision and pattern recognition . 2016. Training Detector OCR Detections Network (i.e. Extracted Text Extraction YOLO, SSD) Acknowledgement goes to BMW ITRC for partnering on this endeavor and providing the data for experimentation Luckow, Andre, et al. "Deep learning in the automotive industry: Applications and tools." Big Data (Big Data), 2016 IEEE International Conference on . IEEE, 2016.

Perception + Control Integrating perception with reinforcement learning

Autonomous Driving in CARLA Action (Steering / Acceleration) Combine Feature Maps Rewards Reinforcement Learning Agent Environment “Approaching Stop Sign” (Carla) “Light is Yellow” “Car stopped at intersection” State (RGB Image) Segmentation Traffic Interpretation Object Detection

Autonomous Driving in CARLA

Planning Using reinforcement learning to explore environments

Neural Network Based Planning VIN GVIN 16 x 16 2D Maze New York City Street Map (13K intersections) A. Tamar, et al. Value Iteration Networks . NIPS, 2016. S. Niu, et al. Generalized Value Iteration Network: Life Beyond Lattices . 32nd AAAI, 2018.

Neural Network Based Planning G S Social network reasoning Navigation Knowledge querying Network routing

Neural Network Based Planning (GVIN) Start Goal

Neural Network Based Planning (GVIN) Testing Training Testing 10 nodes 2642 nodes 5069 nodes S. Niu, et al. Generalized Value Iteration Network: Life Beyond Lattices . 32nd AAAI, 2018.

Distributed Computing Scaling beyond a single HPC cluster

Clemson University’s Palmetto Cluster

Is One HPC Cluster Enough? www.nlm.nih.gov Image credit: http://harborresearch.com/connected-vehicles-rise-transportation-ecosystems/

Scientific Data Analysis at Scale (SciDAS) NSF Award No. 1659300

Scientific Data Analysis at Scale (SciDAS) NSF Award No. 1659300

Scientific Data Analysis at Scale (SciDAS) NSF Award No. 1659300

Scientific Data Analysis at Scale (SciDAS) NSF Award No. 1659300

Scientific Data Analysis at Scale (SciDAS) NSF Award No. 1659300

Wrap Up

Thank you! Questions?