Trivedi_IROS-PPNIV20 2 Seeking Full Autonomy, 2003-2009 [CMU] 2014 - - PDF document

▶
trivedi iros ppniv20
SMART_READER_LITE
LIVE PREVIEW

Trivedi_IROS-PPNIV20 2 Seeking Full Autonomy, 2003-2009 [CMU] 2014 - - PDF document

Dec 20, 2022 238 likes •393 views

Safe Autonomous Driving and Humans: Issues and Prospects Mohan M. Trivedi LISA: Laboratory for Intelligent and Safe Automobiles University of California at San Diego http://cvrr.ucsd.edu/LISA October 2020 Safe Autonomous Driving and Humans:

slide-1
SLIDE 1

Trivedi_IROS-PPNIV20

1

Safe Autonomous Driving and Humans:

Issues and Prospects

Mohan M. Trivedi LISA: Laboratory for Intelligent and Safe Automobiles University of California at San Diego http://cvrr.ucsd.edu/LISA October 2020

Safe Autonomous Driving and Humans:

Issues and Prospects

Outline:

  • Celebrating Accomplishments of the PPNIV community
  • A brief (rear view) look: 1980s till 2015
  • Recognition of some critical elements
  • Age of Safe Autonomous Driving
  • Making of Safe AV: Understanding and Predicting Human behavior
  • Exciting journey continues !
slide-2
SLIDE 2

Trivedi_IROS-PPNIV20

2 Vision for Intelligent Vehicles: Past 1980-2000

Ernst Dickmanns, "The development of machine vision for road vehicles in the last decade." IEEE Intelligent Vehicles Symposium, 2002

Vision for Intelligent Vehicles: Past 1980-2000

Todd Jochem, Dean Pomerleau, Charles Thorpe, "Vision Guided Lane Transition," IEEE Intelligent Vehicles, 1995.

slide-3
SLIDE 3

Trivedi_IROS-PPNIV20

3

[CMU]

Seeking Full Autonomy, 2003-2009

2014 Birth of a new age ?

slide-4
SLIDE 4

Trivedi_IROS-PPNIV20

4

Self Driving Cars 2015 Quest for Fully Autonomous Driving

Seen this?

slide-5
SLIDE 5

Trivedi_IROS-PPNIV20

5

Seniors and Autonomous Driving Seniors and Autonomous Driving

June 2017

What happens if the vehicle refuses to let go ?

What happens if the vehicle doesn’t know it made a mistake?

What happens if the vehicle makes a mistake?

Towards Human-Centered Autonomous Driving

slide-6
SLIDE 6

Trivedi_IROS-PPNIV20

6

Does the vehicle understand state, preferences, intentions, abilities of humans in the vehicle? Does the vehicle understand state, intentions, abilities of humans around the vehicle? Does the vehicle understand state, intentions, abilities of humans driving surrounding vehicles?

Towards Human-Centered Autonomous Driving

Does the vehicle understand state, intentions, abilities of surrounding vehicles?

Humans in vehicle cabin Humans in surround vehicles Humans around vehicle

Safe to deploy airbag? Distracted driver?

Noticed pedestrian? Pedestrian intent?

Pedestrian trajectory? New traffic rules? Distracted pedestrian?

My neighbor’s intent? Acknowledge right of way? Distracted neighbor? Ready to take over? Hands on wheel?

Human-Centered Autonomous Driving: LISA Research Agenda

Ohn-Bar, Trivedi, Humans in the Age of Self-Driving Vehicles, IEEE Trans. Intelligent Vehicles, 2016.

LISA Publications http://cvrr.ucsd.edu/publications/index.html

slide-7
SLIDE 7

Trivedi_IROS-PPNIV20

7

Holistic Distributed Cognitive Systems Perspective:

Learning from Naturalistic Driving Studies, Predictive, Attentive, Holistic Systems

Long-Term Goals: Human cohabitation with intelligent robots Open Issues:

Fail-safe, Control transitions, Trustworthy, Performance Metrics, standards, evaluations, multi-agents, cooperation, Reliable communication links, security, etc. etc.

Big Picture: Safe, Stress-free, Efficient, Enjoyable Driving/Riding

LISA Research: Four Points

LISA Publications http://cvrr.ucsd.edu/publications/index.html

LISA-T: for Safe Autonomous Driving

slide-8
SLIDE 8

Trivedi_IROS-PPNIV20

8

Key Research Contributors

Akshay Rangesh Kevan Yuen Nachiket Deo

LISA-T: for Safe Autonomous Driving LISA-T: for Safe Autonomous Driving

slide-9
SLIDE 9

Trivedi_IROS-PPNIV20

9

Continuous Situational Awareness

A Rangesh, N Deo, K Yuen, K Pirozhenko, M Trivedi, H Toyoda, P Gunaratne, "Exploring the Situational Awareness of Humans inside Autonomous Vehicles," IEEE International Conference on Intelligent Transportation Systems 2018

Continuous Situational Awareness

Ashish Tawari, Andreas Mogelmose, Sujitha Martin, Thomas Moeslund, and Mohan M. Trivedi, "Attention Estimation by Simultaneous Analysis of Viewer and View," IEEE Intelligent Transportation Systems Conference, (ITSC2014), Oct. 2014

slide-10
SLIDE 10

Trivedi_IROS-PPNIV20

10

Dynamics of Driver Gaze: Explorations in Behavior Modeling and Activity Prediction

Control Transitions in Autonomous Vehicles

Control needs to be transferred to driver during failure modes To determine when and how to alert the driver, we need to continuously estimate readiness to take-over

A Rangesh, N Deo, K Yuen, K Pirozhenko, M Trivedi, H Toyoda, P Gunaratne, "Exploring the Situational Awareness of Humans inside Autonomous Vehicles," IEEE International Conference on Intelligent Transportation Systems 2018

Cues for take-over readiness:

Driver gaze activity:

  • Where is the driver

looking?

  • Are they

situationally aware? Driver hand activity:

  • How close are the driver’s

hands to vehicle controls?

  • What activity are their

hands performing?

  • What object are they

interacting with Driver foot activity:

  • How close are the driver’s

feet to vehicle controls?

Control Transitions in Autonomous Vehicles

  • N. Deo, M. Trivedi. “Looking at the Driver/Rider to Predict Take-Over Readiness." IEEE Trans Intelligent Vehicles, 2019. UCSD Invention

disclosure 2019-139 A Rangesh, N Deo, K Yuen, K Pirozhenko, M Trivedi, H Toyoda, P Gunaratne, "Exploring the Situational Awareness of Humans inside Autonomous Vehicles," IEEE International Conference on Intelligent Transportation Systems 2018

slide-11
SLIDE 11

Trivedi_IROS-PPNIV20

11

Driver Gaze Classification, Hand Localization / Classification

Classify Window at Wrist

  • On Wheel
  • Hover Wheel
  • On Lap
  • Radio
  • In Air
  • Cupholder
  • Phone
  • No Phone

Looking at Hands

Kevan Yuen and Mohan M. Trivedi, "Looking at Hands in Autonomous Vehicles:A ConvNet Approach using Part Affinity Fields," IEEE Transactions on Intelligent Vehicles, 2020.

Control Transitions in Autonomous Vehicles

  • N. Deo, M. Trivedi. “Looking at the Driver/Rider to Predict Take-Over Readiness." IEEE Trans Intelligent Vehicles, 2019.

UCSD Invention disclosure 2019-139

Observable Readiness Index (ORI) Estimation

slide-12
SLIDE 12

Trivedi_IROS-PPNIV20

12

Control Transitions in Autonomous Vehicles

  • N. Deo, M. Trivedi. “Looking at the Driver/Rider to Predict Take-Over Readiness." IEEE Trans Intelligent Vehicles, 2019.

UCSD Invention disclosure 2019-139

Observable Readiness Index (ORI) Estimation

Take-over Readiness Estimation Take-over Readiness Estimation

  • N. Deo, M. Trivedi. “Looking at the Driver/Rider to Predict Take-Over Readiness." IEEE Trans Intelligent Vehicles, 2019.

UCSD Invention disclosure 2019-139 A Rangesh, N Deo, K Yuen, K Pirozhenko, M Trivedi, H Toyoda, P Gunaratne, "Exploring the Situational Awareness of Humans inside Autonomous Vehicles," IEEE International Conference on Intelligent Transportation Systems 2018

Observable Readiness Index (ORI) Estimation

slide-13
SLIDE 13

Trivedi_IROS-PPNIV20

13

Take-over Readiness Estimation

  • N. Deo, M. Trivedi. “Looking at the Driver/Rider to Predict Take-Over Readiness." IEEE Trans Intelligent Vehicles, 2019.

UCSD Invention disclosure 2019-139 A Rangesh, N Deo, K Yuen, K Pirozhenko, M Trivedi, H Toyoda, P Gunaratne, "Exploring the Situational Awareness of Humans inside Autonomous Vehicles," IEEE International Conference on Intelligent Transportation Systems 2018

Rider Activity Correlations with Observable Readiness Index (ORI)

Take-over Readiness Estimation Take-over Readiness Estimation

N Deo, N Meoli, A Rangesh M. Trivedi, "On Control Transitions in Autonomous Driving: A Framework and Analysis for Characterizing Scene Complexity," ICCV Workshop on Autonomous Driving, 2019.

Exploring Control Transition and Driving Scene Complexity

Take-over Readiness Estimation

slide-14
SLIDE 14

Trivedi_IROS-PPNIV20

14

Takeover Time Prediction for Autonomous Vehicles: A Machine Learning Approach, UCSD Invention SD2021-070, 2020

Take over Time Prediction Safe Autonomous Driving (AD) => Autonomous Vehicles + Humans

AD = Distributed Cognitive Systems: Human-Vehicle Teams

Research Explorations:

  • Multiple Intelligent Agents,
  • Holistic Situation Perception with Multimodal Sensors
  • Understanding Behavior and Interactions
  • Predicting Intentions
  • Continuous Risk Assessment
  • Smooth/Safe Control Transitions, Fail-safe operation modes,
  • Large Naturalistic Driving Studies and Sharable Datasets
  • Evaluations, Metrics, and Benchmarks
  • Reliability, Robustness and Scalability

Thanks !

LISA Publications http://cvrr.ucsd.edu/publications/index.html

Safe Autonomous Driving: Exciting journey continues !