Understand Basketball Games 2018.6.15 Sports Videos Large - - PowerPoint PPT Presentation

Understand Basketball Games

2018.6.15 吴浩贤 朱⽂斈韬

Sports Videos

Large quantity, high quality Practical utility Stereotypical

Sports Videos

Stereotypical:

Sports Videos

Stereotypical:

[Pass, Score(2-Pointer)] [Pass, Pass, Score(3-Pointer)] [Pass, Pass, Score(3-Pointer)] [Pass, Score(2-Pointer)]

Sports Videos

Stereotypical:

Sports Videos

Recognition

Google Basketball Dataset

• 100+ GB Video on Youtube
• 250+ NCAA Basketball games from 1988 to 2011
• 14,000+ Event annotations (Endpoints)
• Player bounding boxes (Optional)
• Event classes: 11→7
• Free Throw Made/Miss
• 2-pointer Made/Miss
• 3-Pointer Made/Miss
• Steal

http://basketballattention.appspot.com/dataset_browser.html Detecting events and key actors in multi-person videos, Vignesh Ramanathan, Jonathan Huang, Sami Abu-El-Haija, Alexander Gorban, Kevin Murphy, Li Fei-Fei, CVPR 2016, https://arxiv.org/abs/1511.02917

Google Basketball Dataset

Challenges:

• Low resolution & noisy
• Imbalanced categories
• Variant person number in a frame

http://basketballattention.appspot.com/dataset_browser.html Detecting events and key actors in multi-person videos, Vignesh Ramanathan, Jonathan Huang, Sami Abu-El-Haija, Alexander Gorban, Kevin Murphy, Li Fei-Fei, CVPR 2016, https://arxiv.org/abs/1511.02917

Basic Idea

Event Label

Long-term Recurrent Convolutional Networks

Long-term Recurrent Convolutional Networks for Visual Recognition and Description, CVPR2015 Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, Trevor Darrell

Basic Idea

Event Label

🏁 ⛹

Basic Idea

🏁⛹

Detection and/or Tracking

In your imagination…

Ball Detection

Ball Detection

In dataset…

Ball Detection

球的颜⾊艳，形状相对固定，考虑传统⽅斺法 Canny边缘检测+Hough变换应⽤甩于曲线检测 (x-c1)2+(y-c2)2 = r2 图像⼆亍值化，Canny边缘检测, 在边缘像素点（x,y）上枚举c1,c2 累加在像素点(x,y)下的三元组（c1,c2,r)，检测圆形

快速运动中球的形变，⾊艳变；复杂背景下多个候选圆⽬盯 标。

Ball Detection

• Ball Detection

Ball Detection

YOLO = You Only Look Once

Ball Detection

YOLO = You Only Look Once

Model Architecture

Event Label

🏁 ⛹

Feature Extraction

Frame Feature: CNN ResNet (no top) (2048,)

Feature Extraction

Player Feature: CNN VGG19 (no top) (512,) for player

(1877,) for player

Weighted

(1365,) for player

Concat

spatial histogram with pyramid

(1877,) player feature (2048,) frame feature … LSTM

Model Architecture

In a clip Trajectory Extra Constant Vector As Context

Model Architecture 🏁

⛹

🏁

In a clip Trajectory Extra constant vector as context of sequence

Model Architecture 🏁

⛹

🏁 🏁 🏁 🏁

In a clip Trajectory Extra constant vector as context of sequence

Model Optimization

Bidirectional LSTM: Compute a global(clip-level) context feature for each frame

Model Optimization

Next we use a unidirectional LSTM with extra input to represent the state of the event at time t

Model Optimization

Gradient Clipping

Gradient Explode ❌ Clip the gradient before parameter update

I Goodfellow, The cliff Y Bengio, Deep Learning

Results

Spatial Only LRCN Combined (Context) Combined (Concat) Top_1_acc 0.44 0.35 0.47 0.41 Top_2_acc 0.69 0.59 0.70 0.62

Thank You