Digital Audio 3 4 Sampling 44.1 kHz 48 kHz 96 kHz 192 kHz - PowerPoint PPT Presentation

On Audio-Visual File Formats Summary • digital audio and digital video • container, codec, raw data • different formats for different purposes • audio-visual data transformations Université de Lausanne Option «Archives» Cinémathèque suisse, Penthaz, 2017–01–02 Reto Kromer • AV Preservation by reto.ch 1 2 Digital Audio • sampling • quantisation Digital Audio 3 4

Sampling • 44.1 kHz • 48 kHz • 96 kHz • 192 kHz digitisation = sampling + quantisation 5 6 Quantisation • 16 bit • 24 bit Digital Video 7 8

Digital Video Resolution • resolution • SD 480i / SD 576i • bit depth • HD 720p / HD 1080i • linear vs. logarithmic • 2K / HD 1080p • colour model • 4K / UHD-1 • chroma subsampling • 8K / UHD-2 • illuminant 9 10 Bit Depth Linear vs. Logarithmic • 8 bit «medium grey» • 10 bit • linear: 18% • 12 bit • «logarithmic»: 50% • 16 bit 11 12

Colour Model • XYZ • RGB / R ′ G ′ B ′ / CMY / C ′ M ′ Y ′ • Y ′ IQ / Y ′ UV / Y ′ D B D R • Y ′ C B C R / Y ′ C O C G • Y ′ P B P R 13 14 RGB24 00000000 11111111 00000000 00000000 00000000 00000000 11111111 00000000 00000000 00000000 00000000 11111111 00000000 11111111 11111111 11111111 11111111 00000000 11111111 11111111 11111111 11111111 00000000 11111111 15 16

Chroma Subsampling • 4:4:4 • 4:2:2 • 4:2:0 / 4:1:1 17 18 4:4:4 4:2:2 ● ● ● ● ● ● ● ● ● ○ ● ○ ● ○ ● ○ 4 2 ● ● ● ● ● ● ● ● ● ○ ● ○ ● ○ ● ○ 4 2 ● ● ● ● ● ● ● ● ● ○ ● ○ ● ○ ● ○ ● ● ● ● ● ● ● ● ● ○ ● ○ ● ○ ● ○ 19 20

4:2:0 4:1:1 ● ○ ● ○ ● ○ ● ○ ● ○ ○ ○ ● ○ ○ ○ 2 1 ○ ○ ○ ○ ○ ○ ○ ○ ● ○ ○ ○ ● ○ ○ ○ 0 1 ● ○ ● ○ ● ○ ● ○ ● ○ ○ ○ ● ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ● ○ ○ ○ ● ○ ○ ○ 21 22 Illuminant • D50 • D55 • D65 • D75 23 24

File Structure 25 26 File Structure Audio-Visual Container audio-visual container (wrapper) • MP4 • MOV audio codec video codec • AVI • MXF • Matroska (MKV) audio data video data • Flash 27 28

Audio Codec Video Codec • WAVE • H.264 • BWF • ProRes 422 • AIFF • DNxHD • FLAC • FFV1 • AAC • MP3 29 30 Audio Data • mono (1.0) • stereo (2.0) Data is anything • 5.1 (surround) but «raw». 31 32

Video Data «Image Codec» • rgb48le • TIFF • yuv444p16le • DPX • rgb24 • JPEG 2000 • yuv422p10le • OpenEXR • uyvy422 • FFV1 • yuv420p 33 34 «Image Container» What is inside my DPX? • folder • log neg encoding • tar • log RGB encoding or quasi-log encoding • MXF • gamma encoding or power function encoding • Motion JPEG • scene-linear encoding • Matroska 35 36

Principles • The archive must be able to handle the file formats it holds. File Formats • open source • simple to use and well documented • widely used by the community 37 38 Different Purposes Elena Rossi-Snook: archive master format: Archiving without access ➔ for preservation mezzanine format: isn’t preservation, ➔ for professional use in postproduction it’s hoarding. dissemination formats: ➔ for widely spreading and easy access 39 40

Archive Master Mezzanine film video • 2K, folder, TIFF, RGB, 4:4:4, 16 bit • HD, Apple ProRes 422 HQ • 2K, MXF, DPX, R ′ G ′ B ′ , 4:4:4, 10 bit • HD, Avid DNxHD 175x video • HD, AVI, Y ′ C B C R , 4:2:2, 10 bit audio • HD, Matroska, FFV1, Y ′ C B C R , 4:2:2, 10 bit • BWF, 48 kHz, 24 bit • FLAC, 48 kHz, 24 bit audio • BWF, 96 kHz, 24 bit • FLAC, 96 kHz, 24 bit 41 42 Dissemination Archive Master & Mezzanine audio and video combined film • 2K, Matroska, FFV1, R ′ G ′ B ′ , 4:4:4, 16 bit • MP4 video video • HD, Matroska, FFV1, Y ′ C B C R , 4:2:2, 10 bit • SD, H.264, yuv420p • HD, H.264, yuv420p, «lossy» audio • Matroska, FLAC, 96 kHz, 24 bit audio • AAC, 44.1 kHz, 16 bit 43 44

Transformations Data Transformations demultiplex multiplex encode decode filter 47 45 XYZ [M] tristimulus 48 kHz 24 bit RGB transfer function MOV AAC R ′ G ′ B ′ [M] colour diff. encode Audio Exemple Y ′ C B C R 4:4:4 subsampling filter demultiplex multiplex encode decode filter 48 46 Y ′ C B C R 4:2:2 interpolation filter Y ′ C B C R 4:4:4 [M] colour diff. decode 44.1 kHz 16 bit MP4 WAVE R ′ G ′ B ′ transfer function RGB [M] tristimulus XYZ

Video Exemple Audio-Visual Exemple MOV MOV demultiplex demultiplex WAVE ProRes 422 HQ ProRes 422 HQ decode decode 48 kHz 24 bit yuv422p10le yuv422p10le filter filter 44.1 kHz 16 bit yuv420p encode encode yuv420p AAC H.264 H.264 multiplex multiplex MP4 MP4 49 50 Acknowledgements AV Preservation by reto.ch • Swiss Federal Institute of Technology chemin du Suchet 5 1024 Écublens • Massachusetts Institute of Technology Switzerland • Kinemathek Lichtspiel, Bern Web: reto.ch Twitter: @retoch • Charles Poynton Email: info@reto.ch • Dave Rice & Misty De Meo • Agathe Jarczyk & David Pfluger 51 52