Automatic Scenario Generation for Testing and Training Self-driving - - PowerPoint PPT Presentation

▶
automatic scenario generation for testing and training
SMART_READER_LITE
LIVE PREVIEW

Automatic Scenario Generation for Testing and Training Self-driving - - PowerPoint PPT Presentation

Jun 08, 2023 126 likes •538 views

Automatic Scenario Generation for Testing and Training Self-driving Cars Adrien Treuille Zoox / Carnegie Mellon Take 1: Scenario Description Format Design Constraints Drive all simulation modules. Confidential 17 Design Constraints

slide-1
SLIDE 1

Automatic Scenario Generation for Testing and Training Self-driving Cars

Adrien Treuille

Zoox / Carnegie Mellon

slide-2
SLIDE 2

Take 1:

Scenario Description Format

slide-3
SLIDE 3

17 Confidential

Design Constraints

  • Drive all simulation modules.
slide-4
SLIDE 4

18 Confidential

Design Constraints

  • Drive all simulation modules.
  • Convert from real world ->

synthetic data

slide-5
SLIDE 5

19 Confidential

Design Constraints

  • Drive all simulation modules.
  • Convert from real world ->

synthetic data

  • Generate data using an

artist

entity { name: "hero" body { pose { track { id: 110100021, s: 1.5, t: 1.5, } } } hero_vehicle {} } dispatch_command { pose { track { id: 140100161, s: 0.5, t: -1.5, } }

  • bjective: PICKUP,

}

slide-6
SLIDE 6

20 Confidential

  • map

| Synthetic

Scenario Definition Format

World

slide-7
SLIDE 7

22 Confidential

  • map

| Synthetic

  • entities

Scenario Definition Format

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

World

slide-8
SLIDE 8

23 Confidential

  • map

| Synthetic

  • entities

Scenario Definition Format

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

slide-9
SLIDE 9

25 Confidential

Cons:

  • Creating the data is extremely

time consuming

  • ...and limiting!

Recap : Scenario Definition Format

Pros:

  • Drive all simulation modules.
  • Convert from real world ->

synthetic data

  • Generate data using an artist
  • Use computers to generate a

ton of tricky data!

slide-10
SLIDE 10

Take 2:

Scenario Description Language

slide-11
SLIDE 11

30 Confidential

A Combinatorial Perspective

SDF Space

Meaningful Scenario

slide-12
SLIDE 12

31 Confidential

A Combinatorial Perspective

SDF Space

Meaningful Scenario

slide-13
SLIDE 13

32 Confidential

A Combinatorial Perspective

SDF Space

Meaningful Scenario Meaningless Scenario

slide-14
SLIDE 14

33 Confidential

A Combinatorial Perspective

SDF Space

How can we discover (iterate over) just meaningful scenarios?

slide-15
SLIDE 15

36 Confidential

Design Constraints

  • Understandable to Product

Managers / Regulators

  • Compiles to SDF
  • Combinatorial in Nature

SDF Space

slide-16
SLIDE 16

37 Confidential

Design Constraints

  • Understandable to Product

Managers / Regulators

  • Compiles to SDF
  • Combinatorial in Nature
  • Works on real maps.
slide-17
SLIDE 17

38 Confidential

Design Constraints

  • Understandable to Product

Managers / Regulators

  • Compiles to SDF
  • Combinatorial in Nature
  • Works on real maps.

color point clouds

(movies)

slide-18
SLIDE 18

39 Confidential

Design Constraints

  • Understandable to Product

Managers / Regulators

  • Compiles to SDF
  • Combinatorial in Nature
  • Works on real maps.
  • Can be short!
slide-19
SLIDE 19

40 Confidential

  • map

| Synthetic | Real World

  • entities

Scenario Definition Language

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

| Real World

slide-20
SLIDE 20

41 Confidential

  • map

| Synthetic | Real World

  • entities

Scenario Definition Language

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

  • follow_road
  • follow_entity

| Real World

slide-21
SLIDE 21

43 Confidential

  • distance(X,Y) < D
  • in_region(X,R)
  • speed(X) > S
  • speed(X) < S
  • map

| Synthetic | Real World

  • entities

Scenario Definition Language

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

  • G : always (globally)
  • F : in the future
  • R : for release
  • X : next
  • U : until

Linear Temporal Logic Conditions

  • follow_road
  • follow_entity

| Real World

slide-22
SLIDE 22

44 Confidential

  • distance(X,Y) < D
  • in_region(X,R)
  • speed(X) > S
  • speed(X) < S
  • map

| Synthetic | Real World

  • entities

Scenario Definition Language

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

  • G : always (globally)
  • F : in the future
  • R : for release
  • X : next
  • U : until

Linear Temporal Logic Conditions

  • topological
  • follow_road
  • follow_entity
  • outer_products

| Real World

slide-23
SLIDE 23

45 Confidential

Topological Coordinate Systems

slide-24
SLIDE 24

46 Confidential

Topological Coordinate Systems

S T S T T S

slide-25
SLIDE 25

48 Confidential

Outer Products

slide-26
SLIDE 26

49 Confidential

Outer Products

slide-27
SLIDE 27

50 Confidential

Outer Products

slide-28
SLIDE 28

51 Confidential

  • distance(X,Y) < D
  • in_region(X,R)
  • speed(X) > S
  • speed(X) < S
  • map

| Synthetic | Real World

  • entities

Scenario Definition Language

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

  • G : always (globally)
  • F : in the future
  • R : for release
  • X : next
  • U : until

Linear Temporal Logic Conditions

  • topological
  • follow_road
  • follow_entity
  • outer_products

| Real World

slide-29
SLIDE 29

52 Confidential

Scenario Definition Language

Still a very small language!

slide-30
SLIDE 30

53 Confidential

  • map

| Synthetic | Real World

  • entities

Scenario Definition Format

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

slide-31
SLIDE 31

54 Confidential

  • distance(X,Y) < D
  • in_region(X,R)
  • speed(X) > S
  • speed(X) < S
  • map

| Synthetic | Real World

  • entities

Scenario Definition Language

Entities

  • body
  • behavior
  • type

| Static Obstacle | Dynamic Obstacle | Hero Vechile | etc...

  • render properties

Behaviors

  • stop
  • move
  • moveTo

Coordinate Systems

  • global
  • intertial

World

  • G : always (globally)
  • F : in the future
  • R : for release
  • X : next
  • U : until

Linear Temporal Logic Conditions

  • topological
  • follow_road
  • follow_entity
  • outer_products

| Real World

slide-32
SLIDE 32

Future Work

slide-33
SLIDE 33

57 Confidential

  • Optimization over scenario types.

Future Directions

By IkamusumeFan - Own work, CC BY-SA 4.0, https://commons.wiki media.org/w/index.ph p?curid=42043175

slide-34
SLIDE 34

58 Confidential

  • Optimization over scenario types.
  • Precisely characterize the

statistical realism of the scene relative to real data

Future Directions

slide-35
SLIDE 35

59 Confidential

  • Optimization over scenario types.
  • Precisely characterize the

statistical realism of the scene relative to real data

  • Big-data geometry creation for

maps

Future Directions

[1] Chang et al., ShapeNet: An Information-Rich 3D Model Repository arXiv:1512.03012

slide-36
SLIDE 36

60 Confidential

  • Optimization over scenario types.
  • Precisely characterize the

statistical realism of the scene relative to real data

  • Big-data geometry creation for

maps

  • Studying various kinds of

variation

Future Directions

Allen, et al. SIGGRAPH 2003

slide-37
SLIDE 37

61 Confidential

  • Optimization over scenario types.
  • Precisely characterize the

statistical realism of the scene relative to real data

  • Big-data geometry creation for

maps

  • Studying various kinds of

variation

  • Using neural nets to validate the

accuracy of data simulation

Future Directions

slide-38
SLIDE 38

62 Confidential

In Short...

Multi-modal synthesis Captured Camera Data Thousands of Scenarios

slide-39
SLIDE 39

63 Confidential

Thank you!