Automatic Scenario Generation for Testing and Training Self-driving Cars Adrien Treuille Zoox / Carnegie Mellon
Take 1: Scenario Description Format
Design Constraints ● Drive all simulation modules. Confidential 17
Design Constraints ● Drive all simulation modules. ● Convert from real world -> synthetic data Confidential 18
Design Constraints entity { ● Drive all simulation modules. name: "hero" body { ● Convert from real world -> pose { track { id: 110100021, synthetic data s: 1.5, t: 1.5, } ● Generate data using an } } hero_vehicle {} artist } dispatch_command { pose { track { id: 140100161, s: 0.5, t: -1.5, } } objective: PICKUP, } Confidential 19
Scenario Definition Format World - map | Synthetic Confidential 20
Scenario Definition Format World Entities - map - body | Synthetic - behavior - entities - type | Static Obstacle | Dynamic Obstacle | Hero Vechile | etc... - render properties Confidential 22
Scenario Definition Format World Entities Coordinate Systems - map - body | Synthetic - behavior - global - entities - type - intertial | Static Obstacle | Dynamic Obstacle | Hero Vechile | etc... - render properties Behaviors - stop - move - moveTo Confidential 23
Recap : Scenario Definition Format Pros: Cons: ● Drive all simulation modules. ● Creating the data is extremely ● Convert from real world -> time consuming synthetic data ● ...and limiting! ● Generate data using an artist ● Use computers to generate a ton of tricky data! Confidential 25
Take 2: Scenario Description Language
A Combinatorial Perspective Meaningful Scenario SDF Space Confidential 30
A Combinatorial Perspective Meaningful Scenario SDF Space Confidential 31
A Combinatorial Perspective Meaningful Scenario Meaningless Scenario SDF Space Confidential 32
A Combinatorial Perspective How can we discover (iterate over) just meaningful scenarios? SDF Space Confidential 33
Design Constraints ● Understandable to Product Managers / Regulators ● Compiles to SDF ● Combinatorial in Nature SDF Space Confidential 36
Design Constraints ● Understandable to Product Managers / Regulators ● Compiles to SDF ● Combinatorial in Nature ● Works on real maps. Confidential 37
Design Constraints ● Understandable to Product Managers / Regulators ● Compiles to SDF ● Combinatorial in Nature (movies) ● Works on real maps. color point clouds Confidential 38
Design Constraints ● Understandable to Product Managers / Regulators ● Compiles to SDF ● Combinatorial in Nature ● Works on real maps. ● Can be short! Confidential 39
Scenario Definition Language World Entities Coordinate Systems - map - body | Synthetic - behavior - global | Real World | Real World - type - intertial - entities | Static Obstacle | Dynamic Obstacle | Hero Vechile | etc... - render properties Behaviors - stop - move - moveTo Confidential 40
Scenario Definition Language World Entities Coordinate Systems - map - body | Synthetic - behavior - global | Real World | Real World - type - intertial - entities | Static Obstacle | Dynamic Obstacle | Hero Vechile | etc... - render properties Behaviors - stop - move - moveTo - follow_road - follow_entity Confidential 41
Scenario Definition Language World Entities Coordinate Systems - map - body | Synthetic - behavior - global | Real World | Real World - type - intertial - entities | Static Obstacle | Dynamic Obstacle | Hero Vechile | etc... - render properties Behaviors Linear - stop Temporal Logic Conditions - move - moveTo - G : always (globally) - distance(X,Y) < D - follow_road - F : in the future - in_region(X,R) - follow_entity - R : for release - speed(X) > S - X : next - speed(X) < S - U : until Confidential 43
Scenario Definition Language World Entities Coordinate Systems - map - body | Synthetic - behavior - global | Real World | Real World - type - intertial - entities | Static Obstacle - topological | Dynamic Obstacle - outer_products | Hero Vechile | etc... - render properties Behaviors Linear - stop Temporal Logic Conditions - move - moveTo - G : always (globally) - distance(X,Y) < D - F : in the future - follow_road - in_region(X,R) - R : for release - follow_entity - speed(X) > S - X : next - speed(X) < S - U : until Confidential 44
Topological Coordinate Systems Confidential 45
Topological Coordinate Systems T T S S S T Confidential 46
Outer Products Confidential 48
Outer Products Confidential 49
Outer Products Confidential 50
Scenario Definition Language World Entities Coordinate Systems - map - body | Synthetic - behavior - global | Real World | Real World - type - intertial - entities | Static Obstacle - topological | Dynamic Obstacle - outer_products | Hero Vechile | etc... - render properties Behaviors Linear - stop Temporal Logic Conditions - move - moveTo - G : always (globally) - distance(X,Y) < D - F : in the future - follow_road - in_region(X,R) - R : for release - follow_entity - speed(X) > S - X : next - speed(X) < S - U : until Confidential 51
Scenario Definition Language Still a very small language! Confidential 52
Scenario Definition Format World Entities Coordinate Systems - map - body | Synthetic - behavior - global | Real World - type - intertial - entities | Static Obstacle | Dynamic Obstacle | Hero Vechile | etc... - render properties Behaviors - stop - move - moveTo Confidential 53
Scenario Definition Language World Entities Coordinate Systems - map - body | Synthetic - behavior - global | Real World | Real World - type - intertial - entities | Static Obstacle - topological | Dynamic Obstacle - outer_products | Hero Vechile | etc... - render properties Behaviors Linear - stop Temporal Logic Conditions - move - moveTo - G : always (globally) - distance(X,Y) < D - F : in the future - follow_road - in_region(X,R) - R : for release - follow_entity - speed(X) > S - X : next - speed(X) < S - U : until Confidential 54
Future Work
Future Directions ● Optimization over scenario types. By IkamusumeFan - Own work, CC BY-SA 4.0, https://commons.wiki media.org/w/index.ph p?curid=42043175 Confidential 57
Future Directions ● Optimization over scenario types. ● Precisely characterize the statistical realism of the scene relative to real data Confidential 58
Future Directions ● Optimization over scenario types. ● Precisely characterize the statistical realism of the scene relative to real data ● Big-data geometry creation for maps [1] Chang et al., ShapeNet: An Information-Rich 3D Model Repository arXiv:1512.03012 Confidential 59
Future Directions ● Optimization over scenario types. ● Precisely characterize the statistical realism of the scene relative to real data ● Big-data geometry creation for maps ● Studying various kinds of variation Allen, et al. SIGGRAPH 2003 Confidential 60
Future Directions ● Optimization over scenario types. ● Precisely characterize the statistical realism of the scene relative to real data ● Big-data geometry creation for maps ● Studying various kinds of variation ● Using neural nets to validate the accuracy of data simulation Confidential 61
In Short... Multi-modal synthesis Captured Camera Data Thousands of Scenarios Confidential 62
Thank you! Confidential 63
Recommend
More recommend
