A Novel Approach to Integrate Human-in-the-Loop Testing in the - PowerPoint PPT Presentation

Institute of Automotive Engineering A Novel Approach to Integrate Human-in-the-Loop Testing in the Development Chain of Automated Driving on the Example of Automated Lane Change Branko Rogic TU Graz 1

Agenda Institute of Automotive Engineering • Project Introduction • Automated Driving System Development Chain • Automated Lane Change Function & HMI Concept • Human-in-the-Loop Testing with Driving Simulator • Results & Discussion 2

Project Introduction Institute of Automotive Engineering • Project goal: implementation of virtual tools in the development process of automated driving systems • The automated lane change (LC) function is used for demonstration purposes as system under test • Industrial project in cooperation with Magna Steyr Double V-Model for the development of Advanced Driver Assistance Systems (ADAS) 3

Automated Driving Function (ADF) Development Chain Institute of Automotive Engineering Functional concept Survey on driver impressions over tested ADF HMI System (Validation) modeling design Development test- ADSG Tests with scenarios multiple test drivers Benchmark & human driver recording tests System ADSG adaption & parametrisation ADF implementation ADSG – Automated Driving Simulator Graz 4

Automated Lane Change Function & HMI Concept Institute of Automotive Engineering • Lane change algorithm for decision making and trajectory planning based on dynamic traffic (Samiee, 2016) • Parametrisation based on recorded natural test drives with multiple drivers • HMI concept (Quinz, 2017) Manual mode Semi Automated mode Automated mode 5

Human-in-the-Loop Testing Institute of Automotive Engineering Driving Simulator Study • Stationary driving simulator with full vehicle body, 360 ° surrounding view, acoustic and force-feedback simulation • 20 test persons Method • Two-way test concepts: fixed manoeuvres and traffic Front view flow scenario • The fixed maneuvers evaluate driver acceptance with respect to the lane change starting point and its duration (focus function) Side view • The traffic flow test concept gives feedback about overall driver impression regarding usability and user friendliness (focus HMI) Rear view 6

Fixed Manoeuvres Institute of Automotive Engineering • Four repeatable standard manoeuvres with fixed kinematic for all vehicles • Every manoeuvre was driven three times by each test person: Manoeuvre 1 • Manual • Automated variant 1 • Automated variant 2 • Questionnaire after each automated drive Manoeuvre 2 regarding: • Overall performance impression • LC initiation timing • LC duration Manoeuvre 3 Manoeuvre 4 7

Traffic Flow Scenario Institute of Automotive Engineering • Microscopic traffic flow simulation of a three-way highway with calibrated parameters from road measurements (PTV VISSIM) • Simulated driving in “naturalistic” highway traffic • Feedback about the overall driver impression of the LCA system • Focus on HMI • Three LCA modes: • Manual • Semi Automated • Automated • Two-way questionnaire: • NASA Task Load Index • Specific questions regarding system understanding, safety and complexity and driving comfort 8

Fixed Manoeuvre Study Results Institute of Automotive Engineering Initialisation timing [s] LC duration [s] Mean value +/- standard deviation (N=20) 16,1 +/- 9,0 8,3 +/- 4,1 Mean value exl. outliers +/- standard deviation 11,1 +/- 2,8 (N=14, <15s ) 6,7 +/- 1,9 (N=15, <9s) • Initiation timing (calculated over TTC) for most drivers under 15 s • LC duration normally under 9 s • No correlation between manual driving behaviour and rating of automated driving performance � “Driver vs. Passenger” 9

Fixed Manoeuvre Study Results Institute of Automotive Engineering best rate 7 worst rate 1 10

Traffic Flow Scenario Results Institute of Automotive Engineering NASA Task Load Index (TLX) “…subjective , multidimensional assessment tool that rates perceived workload in order to assess a task or other aspects of performance” [Wikipedia] • The questions were adjusted according to the driving task and different LCA modes in highway traffic 13

Traffic Flow Scenario Results Institute of Automotive Engineering • Specific questions regarding system understanding, safety and complexity and driving comfort totally disagree 1 totally agree 7 14

Institute of Automotive Engineering Thank you for your attention 15

References Institute of Automotive Engineering S. Samiee, A. Shahram, K. Reza and A. Eichberger, "Towards a Decision Making Algorithm for Automatic Lane Change Maneuver Considering Traffic Dynamics", Promet - Traffic and Transportation, vol. 28, 2016. 16

A Novel Approach to Integrate Human-in-the-Loop Testing in the - PowerPoint PPT Presentation

Institute of Automotive Engineering A Novel Approach to Integrate Human-in-the-Loop Testing in the Development Chain of Automated Driving on the Example of Automated Lane Change Branko Rogic TU Graz 1 Agenda Institute of Automotive

SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your

IN THE LOOP a scandinavian grassroots approach + 1/30 IN THE LOOP a scandinavian grassroots

Human factors, usability studies and validation testing. Where do you start? How do you approach

Preliminary Transformations Auxiliary Induction Variable Substitution and Loop Normalization

Evolution of a Distributed Live, Virtual, Constructive Environment for Human in the Loop Unmanned

INTEGRATE-HTA Ansgar Gerhardus for the INTEGRATE-HTA project team INTEGRATE-HTA Aim of

The MApUCE project: an interdisciplinary approach to integrate 1.

Optimization in the Loop Implementing and Testing Scheduling Algorithms with SimuLTE Antonio

closed loop under pulsed loads B. Rousset, C.Hoa, B. Lagier, R.Vallcorba 0-D approach to predict

Classification according to underlying testing approach Structural testing Coverage of a

Chapter 10 Retrospective on Unit Testing Software Testing: A Craftsman s Approach, 4 th

1 Exploratory testing approach is seldom considered in our test planning as it is the outlier of

Integrated Family Services Integrate Agency of Human Service efforts across the 6 departments

Human-in-the-loop RL Emma Brunskill CS234 Spring 2017 From here . to

Our approach to open source in the cloud Enable Integrate Release Contribute Engineering

Coarse-Grained Parallelism Variable Privatization, Loop Alignment, Loop Fusion, Loop

Bringing Self-Regulated Learning to Classrooms through Research Practice Partnerships Nancy E.

DECISION MAKING WITH INCOMPLETE INFORMATION MARC C. CANELLAS PH.D. DISSERTATION DEFENSE SCHOOL

Introduction to Quantitative Research and Program Evaluation Methods Dennis A. Kramer II, PhD

State-of-the-Art and Future Trends in Testing of Active Safety Systems Empirical Study Results

Comparison of Transportation Comparison of Transportation Finance Models Finance Models Ray D.

Disclaimer The information provided in this presentation is consistent with the current policies

Introducing Nature Climate Change, Nature Energy , and our Editorial Process Jenn Richler, Ph.D.

How to publish in high impact journals? Thomas Voets Laboratory of Ion Channel Research &

A Novel Approach to Integrate Human-in-the-Loop Testing in the - PowerPoint PPT Presentation

Institute of Automotive Engineering A Novel Approach to Integrate Human-in-the-Loop Testing in the Development Chain of Automated Driving on the Example of Automated Lane Change Branko Rogic TU Graz 1 Agenda Institute of Automotive

SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your

IN THE LOOP a scandinavian grassroots approach + 1/30 IN THE LOOP a scandinavian grassroots

Human factors, usability studies and validation testing. Where do you start? How do you approach

Preliminary Transformations Auxiliary Induction Variable Substitution and Loop Normalization

Evolution of a Distributed Live, Virtual, Constructive Environment for Human in the Loop Unmanned

INTEGRATE-HTA Ansgar Gerhardus for the INTEGRATE-HTA project team INTEGRATE-HTA Aim of

The MApUCE project: an interdisciplinary approach to integrate 1.

Optimization in the Loop Implementing and Testing Scheduling Algorithms with SimuLTE Antonio

closed loop under pulsed loads B. Rousset, C.Hoa, B. Lagier, R.Vallcorba 0-D approach to predict

Classification according to underlying testing approach Structural testing Coverage of a

Chapter 10 Retrospective on Unit Testing Software Testing: A Craftsman s Approach, 4 th

1 Exploratory testing approach is seldom considered in our test planning as it is the outlier of

Integrated Family Services Integrate Agency of Human Service efforts across the 6 departments

Human-in-the-loop RL Emma Brunskill CS234 Spring 2017 From here . to

Our approach to open source in the cloud Enable Integrate Release Contribute Engineering

Coarse-Grained Parallelism Variable Privatization, Loop Alignment, Loop Fusion, Loop

Bringing Self-Regulated Learning to Classrooms through Research Practice Partnerships Nancy E.

DECISION MAKING WITH INCOMPLETE INFORMATION MARC C. CANELLAS PH.D. DISSERTATION DEFENSE SCHOOL

Introduction to Quantitative Research and Program Evaluation Methods Dennis A. Kramer II, PhD

State-of-the-Art and Future Trends in Testing of Active Safety Systems Empirical Study Results

Comparison of Transportation Comparison of Transportation Finance Models Finance Models Ray D.

Disclaimer The information provided in this presentation is consistent with the current policies

Introducing Nature Climate Change, Nature Energy , and our Editorial Process Jenn Richler, Ph.D.

How to publish in high impact journals? Thomas Voets Laboratory of Ion Channel Research &amp;

How to publish in high impact journals? Thomas Voets Laboratory of Ion Channel Research &