AI and Self-Driving Cars Heechul Yun Autonomous Car - PowerPoint PPT Presentation

AI and Self-Driving Cars Heechul Yun

Autonomous Car https://www.latimes.com/business/autos/la-fi-waymo-self-driving-california-20181030-story.html 2

Levels of Automation (SAE J3016) • L1 – Some cars today • L2 – Tesla • L3 – Uber • L4 – Waymo • L5 – None (SAE, "Taxonomy and Definitions for Terms Related to On- Road Motor Vehicle Automated Driving Systems." ) 3

S. Kato, E. Takeuchi, Y. Ishiguro, Y. Ninomiya, K. Takeda, and T. Hamada. ``An Open Approach to Autonomous 4 Vehicles,'' IEEE Micro, Vol. 35, No. 6, pp. 60-69, 2015. Link

Autoware https://www.youtube.com/watch?v=zujGfJcZCpQ 5

Autoware https://github.com/CPFL/Autoware • Open-source software stack for self-driving 6

Autoware https://youtu.be/gq8El7-36z0?t=896 7

Autoware • Limitations – Require detailed 3D map – Require accurate localization (~cm) in the map – Heavily rely on expensive Lidar sensor • Cameras are supplementary – Not the way human drives 8

Bojarski et al., 2016, https://arxiv.org/abs/1604.07316

Pixels to Actions • DNN based supervised learning • Imitating human driving behaviors

How It Works? • Record data  train w/ data  apply in real world • Data = camera input, steering output

http://selfdrivingcars.mit.edu/deeptesla/

End-to-End Control • Deep learning based autonomous systems Image: Prof. Levine, “Deep reinforcement learning via imitation learning”

https://arxiv.org/abs/1901.08567 14

This Week • AI & Deep Learning Basics • Deep Learning Based Self-Driving Cars • Challenges in AI and Real-Time • Papers – End to End Learning for Self-Driving Cars, arXiv, 2016 (Kailani) – DeepPicar: A Low-cost Deep Neural Network-based Autonomous Car, RTCSA, 2018 – An Open Approach to Autonomous Vehicles. MICRO, 2015 (optional) – Autoware on board: enabling autonomous vehicles with embedded systems, ICCPS, 2018 (optional)

AI Resources • Lectures – MIT 6.S094: Deep Learning for Self-Driving Cars – UC Berkeley CS188: Intro to AI – Andrej Karpathy's course on neural networks – Andrew Ng on Coursera – UC Berkeley CS294: Deep Reinforcement Learning – David Silver's course on reinforcement learning • Other useful links – https://gym.openai.com/

AI Resources • Research – BADGR: An Autonomous Self-Supervised Learning- Based Navigation System

DeepPicar https://github.com/mbechtel2/DeepPicar-v2 • End-to-end deep learning: pixels to steering • Using identical DNN with NVIDIA’s DAVE -2 More self-driving videos: https://photos.app.goo.gl/q40QFieD5iI9yXU42 Michael G. Bechtel, Elise McEllhiney, Minje Kim, Heechul Yun. “DeepPicar: A Low -cost Deep Neural Network- based Autonomous Car.” 18 In RTCSA , 2018.

Project: Self-driving RC Car • Your tasks – Build a car • We provide parts (can buy additional parts as needed) – Implement vision based steering • We can provide baseline code – Implement Lidar based emergency braking – Implement vision based traffic signal detection and stop/go – Demo and final report 19

Possible Configuration Safety controller (Basic control + emergency breaking) Lidar HiFive1 rev B Microcontroller Self-Driving Car Camera Raspberry Pi 4 (Linux) Intelligent controller (Vision based steering using DNN) 20

DeepPicar Suite Voice recognition Obstacle detection Traffic sign recognition • Benchmark real-time apps for self-driving cars (on-going) – Vision based steering control (DNN, 250K weights) – Voice recognition (DNN, 750K weights) – Traffic sign recognition (DNN, 1.4K weights) – Obstacle detection & emergency braking (Lidar) • General characteristics – Data and compute intensive – Require efficient & performant computing platform 21

Embedded Platforms • HiFive1 (rev b) board – RISC-V micro-controller – Limited resources/performance – “Bare - metal” programming in C HiFive 1 rev B • Directly access hardware w/o OS • Raspberry Pi 4 – Powerful quad-core ARM CPU – Run fully featured OS (Linux) – Standard PC-like programming environment Raspberry Pi 4 22

Sensors and Actuators

EECS 388 http://www.ittc.ku.edu/~heechul/courses/eecs388/schedule.html

RC Car Platforms

Track

Project Ideas • Reduce the size of the neural network so that it can run on a less powerful computer (e.g., Raspberry pi zero, or hifive 1) ?

Project Ideas • Learning to Drive in a Day https://arxiv.org/pdf/1807.00412.pdf