Condensed Movies: Story Based Retrieval with Contextual Embeddings - - PowerPoint PPT Presentation

condensed movies story based retrieval with contextual
SMART_READER_LITE
LIVE PREVIEW

Condensed Movies: Story Based Retrieval with Contextual Embeddings - - PowerPoint PPT Presentation

Condensed Movies: Story Based Retrieval with Contextual Embeddings Max Bain, Arsha Nagrani, Andrew Brown, Andrew Zisserman Motivation Long term video understanding Context Billy reveals the truth to Louis about the Duke's bet which


slide-1
SLIDE 1

Condensed Movies: Story Based Retrieval

with Contextual Embeddings

Max Bain, Arsha Nagrani, Andrew Brown, Andrew Zisserman

slide-2
SLIDE 2

Motivation

  • Long term video understanding
  • Context

Billy reveals the truth to Louis about the Duke's bet which changed both their lives.

slide-3
SLIDE 3

Motivation

  • Long term video understanding
  • Context

Billy reveals the truth to Louis about the Duke's bet which changed both their lives.

slide-4
SLIDE 4

Motivation

  • Long term video & narrative understanding
  • Context
slide-5
SLIDE 5

Text-Video Retrieval

slide-6
SLIDE 6

Contextual Text-Video Retrieval

slide-7
SLIDE 7
  • 34K captioned clips, from 3.4K movies
  • 1.3K hours video
  • Freely available
  • Pre extracted 6 expert features
  • 400K facetracks, 8K characters
  • Ordered 2 minute clips.

1 2 3 … 8 9 10

Condensed Movies

slide-8
SLIDE 8

Semantic Captions

slide-9
SLIDE 9

Despite illegal moves from the Cobra Kai that bring him to the ground, Daniel (Ralph Macchio) delivers one final kick and wins the championship.

slide-10
SLIDE 10

Overview

  • Context boosts retrieval
  • Contextual task utilizing long term narratives
slide-11
SLIDE 11

Future work

  • Improved temporal aggregation for better 2-minute clip

embeddings

  • Other tasks to test long-term video understanding
slide-12
SLIDE 12

Thank you

  • Paper, dataset, features and code can be found at:
  • https://www.robots.ox.ac.uk/~vgg/research/condensed-movies