Complex Dynamic Scenes Interpretation and Reactive Motion Planning - - PowerPoint PPT Presentation

▶
complex dynamic scenes interpretation and reactive motion
SMART_READER_LITE
LIVE PREVIEW

Complex Dynamic Scenes Interpretation and Reactive Motion Planning - - PowerPoint PPT Presentation

Dec 04, 2023 25 likes •396 views

ParkNav Complex Dynamic Scenes Interpretation and Reactive Motion Planning Robea Project [October 2002- September 2005] Scientific Coordination: Thierry Fraichard e-Motion, Gravir-CNRS & Inria Rhne-Alpes Thierry Fraichard ParkNav

slide-1
SLIDE 1

Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 1/35

ParkNav

Complex Dynamic Scenes Interpretation and Reactive Motion Planning

Robea Project [October 2002- September 2005]

Scientific Coordination: Thierry Fraichard e-Motion, Gravir-CNRS & Inria Rhône-Alpes

slide-2
SLIDE 2

Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 2/35

Introduction

slide-3
SLIDE 3

Introduction

  • Project Summary
  • Scientific Problems
  • Partners
  • Key Component: Dynamic

Map Server

  • 2003 Activity Report
  • 2004 Activity Report

Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 3/35

Project Summary

Automate the driving of a vehicle. . . . . . moving amidst moving objects (vehicles, pedestrians) in a site equipped with video-cameras Joint integrated demonstrator using the Inria Rhône-Alpes testbed

slide-4
SLIDE 4

Introduction

  • Project Summary
  • Scientific Problems
  • Partners
  • Key Component: Dynamic

Map Server

  • 2003 Activity Report
  • 2004 Activity Report

Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 4/35

Scientific Problems

  • Interpretation of complex dynamic scenes with cameras
  • On-line detection, tracking and identification of moving
  • bjects in outdoor environments
  • Accuracy, robustness
  • Reactive motion planning integrating environment dynamicity

and uncertainty on moving objects future behaviour

  • On-line motion computation/adaptation
  • Safety
slide-5
SLIDE 5

Introduction

  • Project Summary
  • Scientific Problems
  • Partners
  • Key Component: Dynamic

Map Server

  • 2003 Activity Report
  • 2004 Activity Report

Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 5/35

Partners

  • e-Motion (ex Sharp), Gravir
  • Project coordination, testbed development supervision
  • Reactive motion planning in highly dynamic environments

(moving objects future behaviour prediction, partial motion planning)

  • Movi, Gravir
  • Multiple cameras calibration, moving object tracking with a

pan-tilt-zoom camera

  • Prima, Gravir
  • Moving object tracking with multiple fixed cameras
  • RIA, Laas-CNRS
  • Reactive motion planning in weakly dynamic environments

(nominal motion deformation)

  • Lagadic (ex Vista), Irisa
  • Moving object tracking with a pan-tilt-zoom camera
slide-6
SLIDE 6

Introduction

  • Project Summary
  • Scientific Problems
  • Partners
  • Key Component: Dynamic

Map Server

  • 2003 Activity Report
  • 2004 Activity Report

Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 6/35

Key Component: Dynamic Map Server

Scene interpretation ↓ ↓ Reactive motion

slide-7
SLIDE 7

Introduction

  • Project Summary
  • Scientific Problems
  • Partners
  • Key Component: Dynamic

Map Server

  • 2003 Activity Report
  • 2004 Activity Report

Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 7/35

2003 Activity Report

  • Testbed development (specification, deployment):
  • Two fixed cameras, one workstation
  • Preliminary version of the Dynamic Map Builder (single

camera, no server)

  • Collaboration of e-Motion, Movi and Prima under the

supervision of Frédéric Hélin (e-Motion)

  • Visible collaboration items
  • Plenary meeting: Gravir, January 2003
  • Visit of Cedric Pradalier (e-Motion) to RIA: experimental

vehicle (Cycab) presentation, June 2003

  • Stay of Olivier Lefèbvre (RIA) with e-Motion: first live

experiments on motion deformation, July 2003

slide-8
SLIDE 8

Introduction

  • Project Summary
  • Scientific Problems
  • Partners
  • Key Component: Dynamic

Map Server

  • 2003 Activity Report
  • 2004 Activity Report

Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 8/35

2004 Activity Report

  • Testbed development
  • Four additional fixed cameras (two wireless), one

pan-tilt-zoom camera, two additional workstations

  • Novel version of the Dynamic Map Server (multi-cameras,

server, Cycab integration)

  • Environment modelling
  • Evaluation of different trackers the outdoor problem. . .
  • Collaboration of e-Motion, Movi and Prima under the

supervision of Fernando De La Rosa,and Eric Boniface (e-Motion)

  • Visible collaboration items
  • Robea workshop: Toulouse, January 2004
  • Plenary meetings: Gravir, March and November 2004
  • Stay of Olivier Lefèbvre (RIA) with e-Motion: second live

experiments on motion deformation, February 2005

  • Joint RIA & e-Motion publication ICRA’04
slide-9
SLIDE 9

Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 9/35

Experimental Testbed

slide-10
SLIDE 10

Introduction Experimental Testbed

  • Camera Layout (Phase 1)
  • Camera Layout (Phase 2)
  • Hardware Architecture

Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 10/35

Camera Layout (Phase 1)

slide-11
SLIDE 11

Introduction Experimental Testbed

  • Camera Layout (Phase 1)
  • Camera Layout (Phase 2)
  • Hardware Architecture

Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 11/35

Camera Layout (Phase 2)

slide-12
SLIDE 12

Introduction Experimental Testbed

  • Camera Layout (Phase 1)
  • Camera Layout (Phase 2)
  • Hardware Architecture

Dynamic Map Server Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 12/35

Hardware Architecture

  • rientable

Station Station hertzien Routeur Capteur Capteur P4 2.4Ghz 512 Mo RAM GNU/Linux RH 9 Capteur Capteur Capteur Capteur Station 512 Mo RAM Xeon 3Ghz GNU/Linux Fedora FC2 512 Mo RAM Xeon 3Ghz GNU/Linux Fedora FC2 Réseau ParkView Parkview Carolus Bistre Capteur

slide-13
SLIDE 13

Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 13/35

Dynamic Map Server

slide-14
SLIDE 14

Introduction Experimental Testbed Dynamic Map Server

  • Moving Object Tracking

(Phase 1)

  • Moving Object Tracking

(Phase 2)

  • Software Architecture
  • Environment Modelling
  • Cycab Integration

Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 14/35

Moving Object Tracking (Phase 1)

slide-15
SLIDE 15

Introduction Experimental Testbed Dynamic Map Server

  • Moving Object Tracking

(Phase 1)

  • Moving Object Tracking

(Phase 2)

  • Software Architecture
  • Environment Modelling
  • Cycab Integration

Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 15/35

Moving Object Tracking (Phase 2)

slide-16
SLIDE 16

Introduction Experimental Testbed Dynamic Map Server

  • Moving Object Tracking

(Phase 1)

  • Moving Object Tracking

(Phase 2)

  • Software Architecture
  • Environment Modelling
  • Cycab Integration

Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 16/35

Software Architecture

slide-17
SLIDE 17

Introduction Experimental Testbed Dynamic Map Server

  • Moving Object Tracking

(Phase 1)

  • Moving Object Tracking

(Phase 2)

  • Software Architecture
  • Environment Modelling
  • Cycab Integration

Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 17/35

Environment Modelling

slide-18
SLIDE 18

Introduction Experimental Testbed Dynamic Map Server

  • Moving Object Tracking

(Phase 1)

  • Moving Object Tracking

(Phase 2)

  • Software Architecture
  • Environment Modelling
  • Cycab Integration

Scene Interpretation Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 18/35

Cycab Integration

slide-19
SLIDE 19

Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 19/35

Scene Interpretation

slide-20
SLIDE 20

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation

  • Movi — Camera

Calibration

  • Movi — Configuration

Estimation

  • Prima — Split-Merge
  • Prima — Multi-Camera

Tracking

  • Lagadic — Cycab Pan-Tilt

Tracking

  • Lagadic — On-Board

Pedestrian Pan-Tilt Tracking Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 20/35

Movi — Camera Calibration

slide-21
SLIDE 21

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation

  • Movi — Camera

Calibration

  • Movi — Configuration

Estimation

  • Prima — Split-Merge
  • Prima — Multi-Camera

Tracking

  • Lagadic — Cycab Pan-Tilt

Tracking

  • Lagadic — On-Board

Pedestrian Pan-Tilt Tracking Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 21/35

Movi — Configuration Estimation

  • Multi-camera views
slide-22
SLIDE 22

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation

  • Movi — Camera

Calibration

  • Movi — Configuration

Estimation

  • Prima — Split-Merge
  • Prima — Multi-Camera

Tracking

  • Lagadic — Cycab Pan-Tilt

Tracking

  • Lagadic — On-Board

Pedestrian Pan-Tilt Tracking Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 22/35

Prima — Split-Merge

slide-23
SLIDE 23

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation

  • Movi — Camera

Calibration

  • Movi — Configuration

Estimation

  • Prima — Split-Merge
  • Prima — Multi-Camera

Tracking

  • Lagadic — Cycab Pan-Tilt

Tracking

  • Lagadic — On-Board

Pedestrian Pan-Tilt Tracking Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 23/35

Prima — Multi-Camera Tracking

slide-24
SLIDE 24

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation

  • Movi — Camera

Calibration

  • Movi — Configuration

Estimation

  • Prima — Split-Merge
  • Prima — Multi-Camera

Tracking

  • Lagadic — Cycab Pan-Tilt

Tracking

  • Lagadic — On-Board

Pedestrian Pan-Tilt Tracking Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 24/35

Lagadic — Cycab Pan-Tilt Tracking

slide-25
SLIDE 25

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation

  • Movi — Camera

Calibration

  • Movi — Configuration

Estimation

  • Prima — Split-Merge
  • Prima — Multi-Camera

Tracking

  • Lagadic — Cycab Pan-Tilt

Tracking

  • Lagadic — On-Board

Pedestrian Pan-Tilt Tracking Reactive Motion Planning Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 25/35

Lagadic — On-Board Pedestrian Pan-Tilt Tracking

slide-26
SLIDE 26

Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 26/35

Reactive Motion Planning

slide-27
SLIDE 27

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 27/35

RIA — Nonholonomic Deformation (Phase 1)

  • RIA & e-Motion joint publication: ICRA 2004
slide-28
SLIDE 28

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 28/35

RIA — Nonholonomic Deformation (Phase 2)

Localisation “jumps” Docking

slide-29
SLIDE 29

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 29/35

e-Motion — Partial Motion Planning

t s s0 sg

slide-30
SLIDE 30

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 29/35

e-Motion — Partial Motion Planning

t s s0 sg t s s0 sg

slide-31
SLIDE 31

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 29/35

e-Motion — Partial Motion Planning

t s s0 sg t s s0 sg t s s0 sg

slide-32
SLIDE 32

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 30/35

e-Motion — Partial Motion Planning

  • Real-time constraint
  • Safety: Inevitable Collision States
  • δ
  • δ
  • δ
  • δ
  • δ
  • δ
  • δ
  • δ
  • −
  • δ
  • −
  • δ
  • δ
  • δ
  • δ
  • −
  • +
  • φ
  • φ
  • φ
  • −
  • φ
  • ∞
  • ∞
  • ∞
  • ∞

−

  • ∆r

Observation & Prediction

slide-33
SLIDE 33

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 31/35

e-Motion — Partial Motion Planning

slide-34
SLIDE 34

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 32/35

e-Motion — Motion Prediction (Phase 1)

Learn THEN Predict scheme — Learning

slide-35
SLIDE 35

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 33/35

e-Motion — Motion Prediction (Phase 1)

Learn THEN Predict scheme — Prediction

slide-36
SLIDE 36

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 34/35

e-Motion — Motion Prediction (Phase 2)

Learn AND Predict scheme — Learning

slide-37
SLIDE 37

Introduction Experimental Testbed Dynamic Map Server Scene Interpretation Reactive Motion Planning

  • RIA — Nonholonomic

Deformation (Phase 1)

  • RIA — Nonholonomic

Deformation (Phase 2)

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Partial Motion

Planning

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 1)

  • e-Motion — Motion

Prediction (Phase 2)

  • e-Motion — Motion

Prediction (Phase 2) Thierry Fraichard ParkNav evaluation, Montpellier, March 31st, 2005 - p. 35/35

e-Motion — Motion Prediction (Phase 2)

Learn AND Predict scheme — Prediction