On-line Video-Editing Challenges in Storisphere Steven Simpson - - PowerPoint PPT Presentation

on line video editing challenges in storisphere
SMART_READER_LITE
LIVE PREVIEW

On-line Video-Editing Challenges in Storisphere Steven Simpson - - PowerPoint PPT Presentation

On-line Video-Editing Challenges in Storisphere Steven Simpson David Hutchison Mu Mu, James Brown, Craig Bojko, Jamie Jellicoe, Ross Wilson Lancaster University School of Computing and Communications Storisphere aims Support


slide-1
SLIDE 1

On-line Video-Editing Challenges in Storisphere

Steven Simpson David Hutchison

Mu Mu, James Brown, Craig Bojko, Jamie Jellicoe, Ross Wilson

Lancaster University School of Computing and Communications

slide-2
SLIDE 2

Storisphere aims

  • Support collaborative storytelling

– Like SourceForge/GitHub, but for video – Aimed at 'hyperlocal TV', community production – Support audio/video, stills, and audio commentary

  • Edit on slim client devices

– Web front-end – Minimal configuration – Server does the hard work – Cache and decode on client for playback

slide-3
SLIDE 3

Editing on slim clients

server player version 1 editing instructions version 2 editing instructions version 3

slide-4
SLIDE 4

Content preparation

  • Separate video and audio tracks.
  • Split each track into independent chunks.

– On GOP boundaries – Using closed GOPs – Publish as static files

  • Write a rush EDL (edit-decision list) to

recombine them into the original.

– One EDL segment per chunk

slide-5
SLIDE 5

EDL Composition

source movies new movie EDL segment segment segment

slide-6
SLIDE 6

Content retrieval

  • Convert chunk-describing segments into

MPEG-4 structural data.

  • Add URI references to chunk files.
  • Send to player.
  • Player parses structural data and fetches

chunks.

  • Cache chunks for re-use!
slide-7
SLIDE 7

Benefits of EDLs

  • All derived content expressed as small text

documents

– Combine segments from separate rushes – Once ingested, no need for further transcoding

  • EDL hierarchy

– Can reference rush EDLs or other derived content – Build up advanced stories in stages – Track origins – Defer choice of resolution until point of playback

slide-8
SLIDE 8

Recursive EDLs

EDLs Rush EDLs

slide-9
SLIDE 9

EDL resolution

A C B A'

200 280

time

50 70 90 110 200 280 260 220 70 10 40 80 140 150 210 180

slide-10
SLIDE 10

Challenges

  • EDLs

– Rational numbers

  • Propagation of prime

factors

  • MPEG-4 format

– Limited effects

  • No visual combination
  • Fixed audio volume

– 32- or 64-bit limits

  • Player

– Missing functionality

  • MPEG-4 edit lists
  • External chunks
  • 64-bit integers

– Inefficient implementation

  • No playback until all chunks

fetched

– Faulty implementation

  • Chunks abandoned
  • Mixed audio frequencies
  • Lingering stills
  • Mixed aspect ratios
slide-11
SLIDE 11

Solutions: Rational numbers

  • Limited set of frame rates

– Numerator of rate becomes denominator of frame

durations: 30000/1001=29.97Hz => 30000 MPEG- 4 timescale

  • Record ideal 'cut' positions.

– Frame boundaries – Editor forbids cutting at arbitrary positions

slide-12
SLIDE 12

Solutions: MPEG-4 format

  • Don't do effects!

– Audio overlays okay, though

  • Lose some accuracy in generation of MPEG-4

edit list.

– Shouldn't be easily perceptible

slide-13
SLIDE 13

Solutions: Player

  • Force use of QuickTime

– Good support for edit lists

  • Use a kerbside stitcher

– Gets round bad/missing implementations of

external chunk fetching

  • Ingest at only one audio quality (44.1kHz)
  • JIT translation of stills into 'very slow' edits
  • Put up with aspect-ratio problems
slide-14
SLIDE 14

Kerbside stitcher

server player chunks integrated file structural data GET range 10000-20000

cache

stitcher

slide-15
SLIDE 15

Better solution: Our own player

  • Requirements

– Needs to be in JavaScript for maximum portability – Needs access to native decoder

  • Byte-based, not file/URI-based
  • Benefits

– Not limited by MPEG-4 – Could do our own effects – Build the stitcher into the player

slide-16
SLIDE 16

Links

http://one.lancs.ac.uk/

Storisphere: collaborative video editing system

(formerly “ONE”)

slide-17
SLIDE 17

Acknowledgements

FIRM http://www.firm-innovation.net/

Storisphere, and especially its MARS component, have developed from the Open Narratives Environment (ONE), conceived by colleagues from BBC Research and Development at MediaCityUK (Michael Sparks and Adrian Woolard), subsequently taken forward by Adam Lindsay and others at Lancaster University. We greatly acknowledge their contribution to the work documented herein.