media decode and 2D composition Daniel Stone http://fooishbar.org - - PowerPoint PPT Presentation

▶

Oct 09, 2023 28 likes •319 views

media decode and 2D composition Daniel Stone http://fooishbar.org not dmabuf ... presentation issues for media formats atomicity timing misc current video state of the art combine EGLImage and OpenMAX display with GLES, or X11's Xv EGL

SLIDE 1

media decode and 2D composition Daniel Stone http://fooishbar.org

SLIDE 2

not dmabuf ...

SLIDE 3

presentation issues for media

SLIDE 4

formats atomicity timing misc

SLIDE 5

current video state of the art combine EGLImage and OpenMAX display with GLES, or X11's Xv

SLIDE 6

EGL is a disaster image_external prevents useful scaling no timing information conversion/filtering not as good as overlays

SLIDE 7

let's use overlays everywhere (or planes, sprites, cursors) ... but how?

SLIDE 8

format negotiation how hard can it be?

SLIDE 9

GStreamer: MIME type and FourCC V4L: FourCC DRM: different FourCC

SLIDE 10

Wayland formats are per-protocol e.g. wl_drm uses DRM FourCC

SLIDE 11

esoteric tiled planar YUV YUV full vs. clamped range RGB colourspaces MIME type is really handy for these

SLIDE 12

atomicity think: scrolling content with video

SLIDE 13

Wayland supports this with subsurfaces 'lock' a surface tree for atomic updates

SLIDE 14

KMS nuclear pageflip implementation extant, not merged next steps unclear

SLIDE 15

timing queuing / feedback / domains

SLIDE 16

accuracy is crucial in media/broadcast

ne frame out every 24 hours

SLIDE 17

queuing provide list of future flips & target times cancel queued flips (requires event)

r supercede / revise?

SLIDE 18

some hardware provides a 'carousel' automatic switching between slots every 16ms cannot reliably implement this w/o carousel frame miss penalty: jump backwards

SLIDE 19

prior art: EGL_NV_present_video target timing ranges arbitrary range of 'video slots'

nly GLX/WGL

SLIDE 20

nothing in Wayland or KMS for this (yet) nuclear pageflip could be a good base?

SLIDE 21

feedback assuming the worst: we will miss sometimes crucial to mitigate impact

SLIDE 22

current DRM vblank events suffice work underway in Wayland EGL has none at all (and no events)

SLIDE 23

domains which time domain do we use? hardware time bases are useless w/o query light queryable timer required for A/V sync

SLIDE 24

currently require CLOCK_MONOTONIC would be brilliant to get hw timers ...

SLIDE 25

wildcard: colourkeying falling back when you open a menu sucks required for STB

SLIDE 26

wildcard: interlaced video ?! perhaps nuclear pageflip again?

SLIDE 27

wildcard: sync/fences discussed tomorrow in Android Graphics need query/event to avoid blocking compositor

SLIDE 28

wildcard: carousels strict 16ms switching between slots

SLIDE 29