IMSC End-to-End Internet Subtitles and Captions Pierre-Anthony - - PowerPoint PPT Presentation

▶
imsc
SMART_READER_LITE
LIVE PREVIEW

IMSC End-to-End Internet Subtitles and Captions Pierre-Anthony - - PowerPoint PPT Presentation

Jan 11, 2024 1.07k likes •1.45k views

W3C TTML Profiles for Internet Media Subtitles and Captions IMSC End-to-End Internet Subtitles and Captions Pierre-Anthony Lemieux, Sandflow Consulting (pal@sandflow.com) Supported by MovieLabs IMSC Developed by the W3C Timed Text Working

slide-1
SLIDE 1

IMSC

End-to-End Internet Subtitles and Captions

W3C TTML Profiles for Internet Media Subtitles and Captions

Pierre-Anthony Lemieux, Sandflow Consulting (pal@sandflow.com) Supported by MovieLabs

slide-2
SLIDE 2

IMSC

Developed by the W3C Timed Text Working Group (TTWG) Application of TTML for subtitles and captions Reduces fragmentation by bringing together multiple profiles of TTML Focal point for Internet subtitles and captions Evolve with worldwide requirements 2

slide-3
SLIDE 3

Working with other formats

3

SMPTE-TT Likely no conversion necessary to IMSC1 CEA 608 SMPTE RP 2052-10 CEA 708 SMPTE RP 2052-11 EBU-TT-D No conversion necessary to IMSC1 EBU STL Via EBU-TT-D (EBU Tech 3360) WebVTT Draft mapping developed by the TTWG

slide-4
SLIDE 4

Adoption

SMPTE ST 2067-2 Interoperable Master Format (IMF) ISO 23000-19 Common media application format (CMAF) for segmented media DVB A174 Digital Video Broadcasting (DVB); TTML Subtitling Systems ATSC A/343 ATSC Standard: Captions and Subtitles CTA WAVE Consumer Technology Association: Web Application Video Ecosystem

4

slide-5
SLIDE 5

End-to-End

Mastering Library CDN Consumer Device Online Service

IMSC

PCM AAC, etc. JPEG 2000, etc... AVC, etc. ISO BMFF (ISO 14496-30) IMF (SMPTE ST 2067)

iOS, web browser, etc. Netflix, etc. Twentieth Century Fox, etc.

CMAF (ISO 23000-19) + DASH (ISO 23009) | HLS MXF (SMPTE ST 377-1)

5

slide-6
SLIDE 6

Demo

6

http://subtitling.irt.de/cmaf/

slide-7
SLIDE 7

Case Study: IMF

Component-based master format (SMPTE ST 2067-2)

IMF Composition Video Audio CEA 608 MXF File IMSC1 Document MP4 File IMSC1 Document Video Audio convert + preserve MP4 File Video (burned-in subs) Audio

7

MXF File CEA 608 IMSC1 CEA 608

CEA 608 distribution

slide-8
SLIDE 8

root container

Basics

Regions defined relative to a root container Text and images flow into regions at specified time coordinates, e.g. HH:MM:SS.fraction Time coordinates are offsets from T=0s

  • Not a timecode timestamp
  • Not tied to video timecode

The author defines regions into which text is flowed

00:00:00.00 00:00:02.20

root container into which text is flowed based on time coordinates. <p region="r1" begin="00:00:02.20">into which text is flowed based on time coordinates.</p> <p region="r1" begin="00:00:00.00" end="00:00:02.20">The author defines regions into which text is flowed </p> <region id="r1" tts:extent="50% 20%" tts:origin="30% 66%"/>

8

slide-9
SLIDE 9

sub #2 sub #1 header

<?xml version="1.0" encoding="UTF-8"?> <tt xml:lang="en" xmlns="http://www.w3.org/ns/ttml" …> <head> <styling> <style xml:id="baseStyle" tts:color="white" tts:textAlign="center"/> <style xml:id="blackBackground" tts:backgroundColor="black"/> <style xml:id="greenBackground" tts:backgroundColor="green"/> <style xml:id="withLinePadding" ebutts:linePadding="0.5c"/> </styling> <layout> <region xml:id="area1" tts:origin="5% 10%" tts:extent="90% 20%" tts:displayAlign="center"/> <region xml:id="area2" tts:origin="5% 70%" tts:extent="90% 20%" tts:displayAlign="center"/> </layout> </head> <body> <div style="baseStyle"> <p region="area1" begin="00:00:01" end="00:00:09"> <span style="greenBackground">Centered text on two lines<br/>without padding.</span> </p> <p region="area2" style="withLinePadding" begin="00:00:01" end="00:00:09"> <span style="blackBackground">Centered text on two lines<br/>with padding.</span> </p> </div> </body> </tt>

Anatomy of an IMSC1 Document

regions timing styles

9

slide-10
SLIDE 10

Flexible styles and writing modes

10

slide-11
SLIDE 11

Text or Image Subtitles

Text

Image (+ text string equivalent)

Once upon a time…

<ittm:altText>Once upon a time…<ittm:altText>

11

slide-12
SLIDE 12

Test Suite

FillLineGap001.ttml Exemplar Render

<?xml version="1.0" encoding="UTF-8"?> <tt xmlns="http://www.w3.org/ns/ttml" ... <body> <div> <p xml:id="subtitle1" region="bottom" begin="00:00:00.000" end="00:00:30.000" style="paragraphStyle"> <span style="spanStyle">##Line gaps##</span><br/> ... <span style="spanStyle">##Line gaps##</span> </p> </div> </body> </tt>

https://github.com/w3c/imsc-tests

12

slide-13
SLIDE 13

A Few Open Source Projects

imscJS JavaScript library for rendering IMSC documents to HTML5 Timed Text Toolkit (ttt) Java-based TTML renderer and validator MP4Box ISO BMFF multiplexer dash.js Reference DASH web player asdcplib Wraps IMSC in MXF

13

Many other projects with some IMSC compatibility, e.g. Shaka Player, Exo Player…

slide-14
SLIDE 14

What is imscJS?

Open source JavaScript library Renders IMSC documents to HTML5 Implements IMSC 1.0.1 today, and IMSC 1.1 soon Used by dash.js (reference DASH player) Supported by MovieLabs and Netflix

https://github.com/sandflow/imscJS

14

slide-15
SLIDE 15

Demos

Sample imscJS web application: http://sandflow.com/imsc1proc/index.html Working with dash.js and MP4Box: http://sandflow.com/public/foms2017-2/CEP150_512kb.htm 15

Browser support for subtitle and caption stylistic features is not perfect.

slide-16
SLIDE 16

IMSC Status Roadmap

IMSC 1 Recommendation [April 2016] IMSC 1.0.1 Recommendation [April 2018] IMSC 1.1 Candidate Recommendation [May 2018]

Publication of IMSC 1.1 Recommendation planned for October 2018

16

slide-17
SLIDE 17

IMSC 1.1

Based on TTML 2 Superset of IMSC 1.0.1

  • IMSC 1.0.1 document is a valid IMSC 1.1 document
  • IMSC 1.1 processor presents an IMSC 1.0.1 document as it would

have been presented by an IMSC 1.0.1 processor

  • A few deprecated features
  • A number of new features…

https://github.com/w3c/imsc-vnext-reqs

17

slide-18
SLIDE 18

Ruby

18

slide-19
SLIDE 19

Tate-Chu-Yoko

19

slide-20
SLIDE 20

Slant

20

slide-21
SLIDE 21

Emphasis Marks

21

slide-22
SLIDE 22

Shadow

22

slide-23
SLIDE 23

Stereoscopic 3D

tts:disparity sets binocular disparity between renderings of a region

  • Positive disparity  perceived behind the plane of the display
  • Negative disparity  perceived in front of the plane of the display

Similar to SMPTE ST 428-7 (D-Cinema) and CEA 708.1 Ignored when not rendering onto a stereoscopic image pair

hello hello Left eye Right eye disparity disparity < 0

23

slide-24
SLIDE 24

HDR in IMSC 1.1

Two options

  • Mapping of SDR RGB colors to HDR presentations
  • Carry PQ images in PNG

Map sRGB onto PQ Map sRGB onto HLG Author-supplied luminance gain (TTML2 tts:luminanceGain and Annex Q.1) Fixed recommended mapping (TTML2 Annex Q.2) Ignored if compositing onto SDR image

24

slide-25
SLIDE 25

Mapping SDR RGB to PQ image

sRGB Linear RGB  80 nits  tts:luminan ceGain Inverse PQ OETF PQ tts:luminanceGain="1.5"

rgb(218,165,32)

absoluteRGB(82.37, 42.21, 0.82)

25

slide-26
SLIDE 26

Mapping sRGB to HLG image

sRGB Linear RGB  0.265 HLG OETF HLG

26

slide-27
SLIDE 27

PQ in PNG

Using the ITU BT.2100 PQ EOTF with the PNG Format (W3C WG Note) Uses the existing iCCP chunk profile name to signal BT.2100 PQ images Graceful processing by legacy decoders with fallback

  • gAMA chunk
  • cHRM chunk
  • embedded ICC profile

Code and examples at https://github.com/sandflow/hdr4png 27

slide-28
SLIDE 28

Thank you!

28