Course outline Week 1 : Introduction to intelligent vehicles, - PDF document

ME400 Intelligent vehicles and intelligent transportation systems (ITS) Week 4 : Vehicle perception and map building Denis Gingras January 2015 1 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 Course outline  Week 1 : Introduction to intelligent vehicles, context, applications and motivations  Week 2 : Vehicle dynamics and vehicle modelling  Week 3: Positioning and navigation systems and sensors  Week4: Vehicular perception and map building  Week 5 : Multi-sensor data fusion techniques  Week 6 : Object detection, recognition and tracking  Week 7: ADAS systems and vehicular control  Week 8 : VANETS and connected vehicles  Week 9 : Multi-vehicular scenarios and collaborative architectures  Week 10 : The future: toward autonomous vehicles and automated driving (Final exam) 2 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 1

Week 4 outline  Brainstorming and introduction  Context and importance of vehicle perception and map building  Perception and map building systems basic architectures  Scene analysis  Range sensors  Radar  Lidar  Sonar  Vision based systems  Cameras  Mono vision 2D scene analysis  Stereo Vision 3D scene analysis  Night vision  Landmarks in automated guided vehicles applications 3 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 Brainstorming Brainstorming Open questions and introductory discussion What is perception ? 4 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 4 2

Brainstorming Brainstorming Open questions and introductory discussion What are the two main purposes of perception in the context of intelligent vehicles? 5 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 5 Brainstorming Brainstorming Open questions and introductory discussion Does perception provide a unique interpretation? 6 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 6 3

Brainstorming Brainstorming Open questions and introductory discussion Name a few meaningful structures we want to extract from the vehicle environment. 7 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 7 Brainstorming Brainstorming Open questions and introductory discussion Name a few sensors for vehicular perception. 8 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 8 4

Brainstorming Brainstorming Open questions and introductory discussion Name a few intelligent vehicle applications which are using perception sensors and data. 9 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 9 Brainstorming Brainstorming Open questions and introductory discussion What kind of objects are we interested in about the road infrastructure? 10 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 10 5

Brainstorming Brainstorming Open questions and introductory discussion What physical quantities can we use to detect objects in the perception process ? 11 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 11 Brainstorming Brainstorming Open questions and introductory discussion Which frequency bands are typically used in vehicular perception? 12 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 12 6

Introduction Perception Intro  We can ask the following question:  "given the sensory reading I am getting, what was the world like to make the sensor give me this reading."  This is what is done in computer vision, for example, where: the sensor (a camera) provides a great deal of information (for  example, 512 x 512 pixels = 262,144 pixels of black & white, or gray levels, or color), and  we need to compute what those pixels correspond to in the real world (i.e., a vehicle, a pedestrian?). 13 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 13 Introduction Perception Intro  Sensors do not provide state/symbols, just signals  A great deal of computation may be required to convert the signal from a sensor into useful state information for the vehicle.  This process bridges the areas of:  electronics,  signal processing, and  computation.  Sensory data processing is challenging and can be computationally intensive and time consuming.  It means the intelligent vehicle needs a brain to do this processing. 14 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 14 7

Introduction Perception Intro Modern approach to perception  Perception in the context of action and the task at hand  Action-oriented perception  Expectation-based perception uses a priori knowledge about the world as constraints on sensor interpretation  Focus-of-attention methods provide constraints on where to look  Perceptual classes partition the world into useful categories 15 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 15 Introduction Perception Intro Local perception and its use in intelligent vehicle applications 16 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 8

Introduction Perception Intro Environment representation Source: Jan Becker, ME Dept., Stanford University, 2014 17 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 Introduction Perception Intro Source: Jan Becker, ME Dept., Stanford University, 2014 18 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 9

Introduction Perception Intro Sensing Obstacles  Obstacle detection is much more difficult than vehicle detection: obstacles can be small, non- metallic, and much harder to see  Obstacles can be stationary or moving (e.g. deer running across the road)  For a passenger car at highway speeds, obstacles need to be detected 100 m ahead. For trucks, the distance is even longer. Source: Moras J et al., A lidar Perception Scheme for Intelligent Vehicle Navigation. 11th Int. Conference on  Obstacle detection is one of the Control, Automation, Robotics and Vision, Dec 2010, Singapour. most challenging tasks for an intelligent vehicle 19 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 Introduction Perception Intro Obstacles on the Road  State DOTs report cleaning up construction debris, fuel spills, car parts, tire carcasses, and so forth.  State highway patrols receive reports of washing machines, other home appliances, ladders, pallets, deer, etc.  A survey commissioned by a company that builds litter-retrieval machines reports 185 million pieces of litter / week.  Rural states report up to 35% of all rural crashes involve animals, mostly deer but also including moose and elk as well as farm animals.  A non-scientific survey indicates that people have hit tire carcasses, mufflers, deer, dogs, even a toilet ! 20 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 10

Introduction Perception Intro General perception process This figure shows how roughly raw energy detected from the environment is converted into a situational understanding of the world around the vehicle. It shows also where noise and errors can be introduced into the system and the modeling assumptions that specify how the information is altered at each step. Categories of errors marked with * are generally referred to as artifacts .) Source: Darms M. et al, Obstacle Detection and Tracking for the Urban Challenge, IEEE Transactions On ITS, Vol. 10, No. 3, Sept. 2009 21 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 Introduction Perception Intro General perception architecture This figure shows the architecture of a typical perception system. It is divided according to the world model into three subsystems: 1) a road estimation subsystem, which generates information about the road structure; 2) a tracking subsystem, which is responsible for generating dynamic obstacle hypotheses; and 3) a static obstacle estimation subsystem, which estimates the location of static obstacles and builds the local static map. Source: Darms M. et al, Obstacle Detection and Tracking for the Urban Challenge, IEEE Transactions On ITS, Vol. 10, No. 3, Sept. 2009 22 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 11

Introduction Perception Intro Example: a perception system architecture for the DARPA Urban Challenge Source: John Leonard et al., A Perception-Driven Autonomous Urban Vehicle, Journal of Field Robotics, 1–48 (2008) Wiley Periodicals, Inc. 23 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 Introduction Perception Intro Sensor features evolution 24 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 12

Introduction Perception Intro Sensors using wave reflection. Directed reflection Bad reflection Diffuse reflection 25 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 25 Introduction Perception Intro Reflectivity of selected objects typically found in vehicular perception Source: Jan Becker, ME Dept., Stanford University, 2014 a: heavily depends on the angle R: through retro-reflectors 26 20-déc.-14 D Gingras – ME470 IV course CalPoly Week 4 13