instance learning
play

Instance Learning Boris Babenko, Ming-Hsuan Yang, Serge Belongie - PowerPoint PPT Presentation

Visual Tracking with Online Multiple Instance Learning Boris Babenko, Ming-Hsuan Yang, Serge Belongie Kelsie Zhao Content Goal Background: Tracking by Detection Previous Work New Tracking Solution MILTrack Online MILBoost


  1. Visual Tracking with Online Multiple Instance Learning Boris Babenko, Ming-Hsuan Yang, Serge Belongie Kelsie Zhao

  2. Content • Goal • Background: Tracking by Detection • Previous Work • New Tracking Solution • MILTrack • Online MILBoost • Experiments & Results

  3. Goal Track one arbitrary object in video, given its location in first frame

  4. Background: Tracking by detection • Frame 1 is labeled, tracker location known

  5. Background: Tracking by detection • Crop one positive and some negative patches near tracker x 1 x 3 x 2 Positive Negative

  6. Background: Tracking by detection • Use patches to train the classifier x 1 x 3 x 2 Positive Negative {(x 1 , 1), (x 2 , 0), (x 3 , 0)} Classifier

  7. Background: Tracking by detection • Frame 2 comes Classifier

  8. Background: Tracking by detection • Calculate classifier response within a range of the old tracker location X Old location Classifier

  9. Background: Tracking by detection • Find the maximum response location X X Old location New location Classifier

  10. Background: Tracking by detection • Move tracker Frame 1 Frame 2

  11. Background: Tracking by detection • Repeat Positive Negative Frame 2

  12. Background: Tracking by detection • Problem: If tracker location is not precise, might select bad training examples

  13. Background: Tracking by detection • Problem: If tracker location is not precise, might select bad training examples Model start to degrade!

  14. Background: Tracking by detection • Problem: If tracker location is not precise, might select bad training examples Model start to degrade! • How to select good training examples?

  15. Previous Work • Solution 1: multiple positive examples around tracker location x 2 x 5 x 4 x 3 x 1 {(x 1 , 1), (x 2 , 1), (x 3 , 1), (x 4 , 0), (x 5 , 0)} Classifier

  16. Previous Work • Solution 1 Might confuse classifier! x 2 x 5 x 4 x 3 x 1 {(x 1 , 1), (x 2 , 1), (x 3 , 1), (x 4 , 0), (x 5 , 0)} Classifier

  17. Previous Work • Solution 2: Multiple Instance Learning (MIL) [Keeler ‘90, Dietterich et al. ‘97]

  18. Previous Work: Multiple Instance Learning x 12 x 13 x 11 x 31 x 21 • Multiple examples in one bag [Keeler ‘90, Dietterich et al. ‘97]

  19. Previous Work: Multiple Instance Learning x 12 X 1 x 13 x 11 X 2 X 3 x 31 x 21 • Multiple examples in one bag [Keeler ‘90, Dietterich et al. ‘97]

  20. Previous Work: Multiple Instance Learning x 12 (X 1 , 1) x 13 x 11 (X 3 , 0) (X 2 , 0) x 31 x 21 • Multiple examples in one bag • One bag one label [Keeler ‘90, Dietterich et al. ‘97]

  21. Previous Work: Multiple Instance Learning x 12 (X 1 , 1) x 13 x 11 (X 3 , 0) (X 2 , 0) x 31 x 21 • Multiple examples in one bag • One bag one label • Bag Positive if at least one example is Positive [Keeler ‘90, Dietterich et al. ‘97]

  22. Previous Work: Multiple Instance Learning x 12 (X 1 , 1) x 13 x 11 (X 3 , 0) (X 2 , 0) x 31 x 21 {(X 1 , 1), (X 2 , 0), (X 3 , 0)} Classifier [Keeler ‘90, Dietterich et al. ‘97]

  23. Previous Work: Multiple Instance Learning MIL training input: 𝒀 𝟐 , 𝑧 1 … 𝒀 𝒐 , 𝑧 𝑜 , ( X 1 , 1) x 12 where, x 13 x 11 𝒀 𝒋 = 𝑦 𝑗1 … 𝑦 𝑗𝑛 , ( X 3 , 0) ( X 2 , 0) 𝑧 𝑗 = 𝑛𝑏𝑦 𝑘 𝑧 𝑗𝑘 x 31 x 21 [Keeler ‘90, Dietterich et al. ‘97]

  24. Previous Work: Multiple Instance Learning MIL training input: 𝒀 𝟐 , 𝑧 1 … 𝒀 𝒐 , 𝑧 𝑜 , ( X 1 , 1 ) x 12 where, x 13 x 11 𝒀 𝒋 = 𝑦 𝑗1 … 𝑦 𝑗𝑛 , ( X 3 , 0) ( X 2 , 0) 𝑧 𝑗 = 𝑛𝑏𝑦 𝑘 𝑧 𝑗𝑘 x 31 x 21 Bag babel is 1 if at least one instance is 1 [Keeler ‘90, Dietterich et al. ‘97]

  25. Now we have training examples! How to train the classifier?

  26. Previous Work: MILBoost • MIL + boosting Train a boosting classifier that maximizes log likelihood of bags 𝑚𝑝𝑕𝑀 = log (𝑞 𝑧 𝑗 𝒀 𝒋 )) 𝑗 where, 𝑞 𝑧 𝑗 𝒀 𝒋 ) = 1 − (1 − 𝑞 𝑧 𝑗 𝑦 𝑗𝑘 )) 𝑘 [Viola et al. ‘05]

  27. Previous Work: MILBoost • MIL + boosting Train a boosting classifier that maximizes log likelihood of bags 𝑚𝑝𝑕𝑀 = log (𝑞 𝑧 𝑗 𝒀 𝒋 )) 𝑗 where, 𝑞 𝑧 𝑗 𝒀 𝒋 ) = 1 − (1 − 𝑞 𝑧 𝑗 𝑦 𝑗𝑘 )) 𝑘 ~1 1 [Viola et al. ‘05]

  28. Previous Work: MILBoost • Problem: need all training examples [Viola et al. ‘05]

  29. Previous Work: MILBoost But in tracking, only current frame available [Viola et al. ‘05]

  30. Previous Work: MILBoost But in tracking, only current frame available Need an online training algorithm for MIL [Viola et al. ‘05]

  31. Main Contribution of this paper • Online-MILBoost: Online training for MIL-based classifier • MILTrack New tracking solution using Online-MILBoost

  32. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  33. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  34. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  35. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  36. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  37. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) Babenko et al., 09

  38. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: r = 5 in authors’ experiment Babenko et al., 09

  39. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} ( X 1 , 1) 2. Use MIL classifier to find new tracker location x 12 x 13 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) x 11 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: r = 5 in authors’ experiment Babenko et al., 09

  40. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 3. 2) Crop Negative examples 𝑌 𝑠, 𝛾 = 𝑦 𝑠 𝑢𝑝 𝛾 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑏𝑥𝑏𝑧 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: 𝛾 = 50 in authors’ experiment Babenko et al., 09

  41. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) x 31 x 21 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} ( X 3 , 0) ( X 2 , 0) 3. 2) Crop Negative examples 𝑌 𝑠, 𝛾 = 𝑦 𝑠 𝑢𝑝 𝛾 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑏𝑥𝑏𝑧 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: 𝛾 = 50 in authors’ experiment Babenko et al., 09

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend