1 End-to-End 3D Multi-Object Tracking and Trajectory Forecasting - PowerPoint PPT Presentation

1

End-to-End 3D Multi-Object Tracking and Trajectory Forecasting Xinshuo Weng*, Ye Yuan*, Kris Kitani Robotics Institute, Carnegie Mellon University European Conference on Computer Vision (ECCV) Workshops * denotes equal contributions 2

Limitation of the Prior Work Sensor Data Limitations 3D Object Detection 1. 3D MOT and trajectory forecasting modules are separately trained without joint optimization 3D Multi-Object → Sub-optimal performance and slow inference speed Tracking (MOT) 2. Errors from 3D MOT results will directly influence the trajectory forecasting module due to the sequential pipeline → Errors in the upstream module cannot be corrected Trajectory Forecasting Predicted future trajectories 3

Our Contributions 1. An End-to-End MOT and trajectory forecasting framework that runs in parallel → Enable joint optimization → Prevent errors in 3D MOT from affecting forecasting 4

Our Approach 3D MOT Shared Feature Learning Edge features 3D MOT Feature head extraction Last frame Current frame Objects trajectories in Forecasting GNN for feature past H frames interaction Feature Trajectory Diversity extraction Detected objects in forecasting Node features sampling current frame head Predicted trajectories in Joint 3D Tracking and Forecasting future T frames 5

Ablation Study 6

Joint 3D MOT and Trajectory Forecasting • Is it useful to do joint optimization? • Add joint optimization with forecasting improves performance on tracking Improvement on 5 out of 6 entries! 3D MOT evaluation without forecasting module 7

Joint 3D MOT and Trajectory Forecasting • Is it useful to do joint optimization? • Add joint optimization with forecasting improves performance on tracking • Add joint optimization with 3D MOT improves performance on forecasting Performance improved after adding MOT! Forecasting evaluation without 3D MOT 8

End-to-End 3D Multi-Object Tracking and Trajectory Forecasting Xinshuo Weng*, Ye Yuan*, Kris Kitani Robotics Institute, Carnegie Mellon University European Conference on Computer Vision (ECCV) Workshops * denotes equal contributions 9