Using Linux Media Controller for Wayland/Weston Renderer � Technology Consulting Company Research, Development & Takanari Hayama Global Standard � taki@igel.co.jp http://www.igel.co.jp/ 1� Technology Consulting Company IGEL Co.,Ltd. �
Agenda � • Wayland/Weston Overview • Porting Weston to R-Car • Why Linux Media Controller Renderer? • Linux Media Controller Framework • V4L2 Renderer Design • Conclusions 2�
WAYLAND/WESTON OVERVIEW � 3�
Weston Architecture � Shell Client Wayland Client � Wayland Client � (e.g. desktop-shell) � Weston � Compositor Core � Compositor Shell (e.g. DRM) � (e.g. desktop) � Renderer (e.g. GL) � Wayland IPC � Local API Call � 4� Confidential �
Weston w/ DRM Backend � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � Qt, EFL, etc. � shell � Client Stub � Server Stub � core � compositor-drm � pixman- gl-renderer � renderer � libgbm/ OpenGL/ES � libpximan � libdrm � User Space Kernel Space GPU Kernel DRM/KMS Driver � Driver � Weston Wayland Other IPC � API Call Components � Components � Components � 5�
Rendering and Composition: Overview (GL-Renderer) � wl_buffer Type � Wayland Client � Weston / DRM Compositor � wl_egl_window Type � CLIENT BUFFERS � FULL SCREEN � SCANOUT BUFFERS � FRAME BUFFERS � wl_buffer Type � gbm_surface Type � wl_buffer Type � DRM/KMS BO Type � wayland protocol � GBM � OpenGL/ES + WL Ext. � OpenGL/ES + wayland_egl � wl_egl_window Type � wl_buffer Type � GPU Driver for Wayland � DRM/KMS � GPU driver for Wayland � DISPLAYING � RENDERING � Software � Hardware � GPU � Display � Wayland/Weston Khronos w/ OSS Hardware Components � Wayland Ext. � Components � Specific � 6�
Rendering and Composition: Window Composition � 3. Import w/ eglCreateImageKHR() � wl_buffer Type � Wayland Client � Weston / DRM Compositor � wl_egl_window Type � CLIENT BUFFERS � FULL SCREEN � SCANOUT BUFFERS � FRAME BUFFERS � 1. Render w/ wl_buffer Type � gbm_surface Type � wl_buffer Type � DRM/KMS BO Type � OpenGL/ES � 5. Compose w/ 6. Set composed OpenGL/ES � wayland protocol � buffers as KMS 2.Commit buffers w/ BOs. � eglSwapBuffers() GBM � OpenGL/ES + WL Ext. � OpenGL/ES + wayland_egl � wl_egl_window Type � 4. Register destination frame buffers allocated wl_buffer Type � with GBM. � GPU driver for Wayland DRM/KMS � GPU driver for Wayland DISPLAYING � RENDERING � Software � Hardware � GPU � Display � Wayland/Weston Khronos w/ OSS Hardware Components � Wayland Ext. � Components � Specific � 7�
Rendering and Composition: Full Screen or Sprite Rendering � 3. Import w/ gbm_bo_import() � wl_buffer Type � Wayland Client � Weston / DRM Compositor � wl_egl_window Type � CLIENT BUFFERS � FULL SCREEN � SCANOUT BUFFERS � FRAME BUFFERS � 1. Render w/ wl_buffer Type � gbm_surface Type � wl_buffer Type � DRM/KMS BO Type � OpenGL/ES � 4. Set composed wayland protocol � buffers as KMS 2.Commit buffers w/ BOs. � eglSwapBuffers() GBM � OpenGL/ES + WL Ext. � OpenGL/ES + wayland_egl � wl_egl_window Type � wl_buffer Type � GPU driver for Wayland DRM/KMS � GPU driver for Wayland DISPLAYING � RENDERING � Software � Hardware � GPU � Display � Wayland/Weston Khronos w/ OSS Hardware Components � Wayland Ext. � Components � Specific � 8�
PORTING WESTON TO R-CAR � 9�
What Are Required? � 1. OpenGL/ES for Wayland/Weston 2. Zero Copy Mechanism for Native Buffer “Typically, hardware enabling includes modesetting/display and EGL/GLES2. On top of that, Wayland needs a way to share buffers efficiently between processes.” http://wayland.freedesktop.org/architecture.html � 10�
Wayland Requirements for OpenGL/ES � Must support the following Native Display Types for • eglGetDisplay(): – wl_display for clients – gbm handle for Weston Must support the following EGL_EXTENSIONs: • – EGL_KHR_image_pixmap – EGL_WL_bind_wayland_display Must support the following Native Pixmap Type for • eglCreateImageKHR(): – EGL_WAYLAND_BUFFER_WL Must support the following Wayland extension APIs: • – eglBindWaylandDisplayWL – eglUnbindWaylandDisplayWL – eglQueryWaylandBufferWL � 11�
Weston for Renesas R-Car � Wayland Client � Weston (Wayland Server) � wayland-egl � EGL with Wayland Extension � Renesas OpenGL/ES � WSEGL for Wayland � Wayland Client Support � Wayland Server Support � wayland-kms � Wayland Client libdrm & Wayland libgbm w/ KMS Stub � libkms � Server Stub � Backend � Wayland/Weston Standard OSS New OSS IPC � API Call Proprietary � Components � Components � Components � 12�
Wayland Composition Revisited � 1. A client creates a wl_surface on the server. 2. The client attach a wl_buffer to the created surface. 3. The client submit the wl_buffer to the server. 4. The server takes the wl_buffer and compose to the screen. All of above should happen in zero-copy manner! � 13�
What is wl_buffer by the way? � • An abstract data type that represents a reference to a pixel buffer. • 2 open source implementations: – wl_shm : wayland standard • Based on Linux shared memory. Not physically contiguous. – wl_drm : Mesa standard • Based on DRI. Possibly physically contiguous. • Weston understands wl_shm only. Wl_drm is Mesa specific. Thus, wl_drm is not handled by Weston, but by Mesa internally. 14�
Which wl_buffer implementation to use? � • Requirements – End-to-end Buffer Zero Copy – Physically Contiguous Memory • wl_drm? – Implementation is too Mesa dependent. • Need more generic implementation. 15�
wl_kms � • KMS BO buffer type. – https://github.com/thayama/wayland-kms – Based on wl_drm in Mesa. • Imports DMABUF via PRIME, a dma-buf interface layer in DRM. – Originally, we used DRM Handle, but we now use DMABUF instead. • Can directly pass video output from V4L2. 16�
Buffer Zero Copying with wl_kms � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � gl-renderer � Client Stub � Server Stub � drm � Wayland Wayland OpenGL/ES � GPU Driver � GPU Driver � User Space � Kernel Space � KMS Driver � KMS BO � 17�
Buffer Zero Copying with wl_kms � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � gl-renderer � Client Stub � Server Stub � drm � Buffer Allocation. Client Wayland Wayland OpenGL/ES � creates rendering surface GPU Driver � GPU Driver � with EGL API. WSEGL allocates memory with KMS BO, and make User Space � avaialable to GPU. � Kernel Space � KMS Driver � KMS BO � 18�
Buffer Zero Copying with wl_kms � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � gl-renderer � Client Stub � Server Stub � drm � Wayland Wayland OpenGL/ES � Rendering to the GPU Driver � GPU Driver � allocated buffer. GPU gets all details needed User Space � about the buffer via WSEGL. � Kernel Space � KMS Driver � KMS BO � 19�
Buffer Zero Copying with wl_kms � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � gl-renderer � Client Stub � Server Stub � drm � Wayland Wayland OpenGL/ES � GPU Driver � GPU Driver � When a client calls User Space � eglSwapBuffers(), WSEGL commits a buffer to the Kernel Space � server via Wayland. The KMS Driver � details of the buffer is DMABUF fd, a stride, a size, and a pixelf ormat. � KMS BO � 20�
Buffer Zero Copying with wl_kms � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � gl-renderer � Client Stub � Server Stub � drm � Wayland Wayland OpenGL/ES � GPU Driver � GPU Driver � When the server receives User Space � the buffer, it imports with eglCreateImageKHR(). � Kernel Space � KMS Driver � KMS BO � 21�
Buffer Zero Copying with wl_kms � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � gl-renderer � Client Stub � Server Stub � drm � Wayland Wayland OpenGL/ES � GPU Driver � GPU Driver � User Space � Kernel Space � KMS Driver � WSEGL gets details of the buffer from wayland- kms, and asks to import the given DMABUF. The buffer is then made KMS BO � available to GPU. � 22�
Buffer Zero Copying with wl_kms � Wayland Client � Weston (Wayland Server) � Wayland Wayland compositor- OpenGL/ES � gl-renderer � Client Stub � Server Stub � drm � Wayland Wayland OpenGL/ES � GPU Driver � GPU Driver � Gl-renderer can now User Space � refer the buffer passed Kernel Space � by the client, and KMS Driver � composes a final output. � KMS BO � 23�
WHY LINUX MEDIA CONTROLLER RENDERER? � 24�
Motivation � • Applications are heading towards more and more GPU intensive. • People want to use GPU for more advanced UI, rather than a simple window composition. – On the other hand, some people want to do more complex composition using GPU. J • GPU Offloading is one way. – https://archive.fosdem.org/2014/schedule/ event/wayland_gpu/ – But, still premature for real products. 25�
Recommend
More recommend
