The MeeGo Multimedia Stack Dr. Stefan Kost Nokia - The MeeGo - - PowerPoint PPT Presentation

the meego multimedia stack
SMART_READER_LITE
LIVE PREVIEW

The MeeGo Multimedia Stack Dr. Stefan Kost Nokia - The MeeGo - - PowerPoint PPT Presentation

The MeeGo Multimedia Stack Dr. Stefan Kost Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe The MeeGo Multimedia Stack MeeGo Intro Architecture Development GStreamer Dr. Stefan Kost Nokia - The MeeGo Multimedia


slide-1
SLIDE 1
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

The MeeGo Multimedia Stack

slide-2
SLIDE 2
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

The MeeGo Multimedia Stack

MeeGo Intro Architecture Development GStreamer

slide-3
SLIDE 3
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Quick MeeGo Intro

  • MeeGo = Moblin + Maemo
  • Linux distribution for CE devices
  • Netbook, Phone (Handset), Connected TV, In-Vehicle
  • http://meego.com
  • Linux Foundation hosted project
slide-4
SLIDE 4
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Architecture

http://meego.com/developers/meego-architecture

slide-5
SLIDE 5
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Architecture - Multimedia

  • Kernel
  • V4l2, Alsa
  • Middleware
  • GStreamer, Pulseaudio, GUPnP
  • API
  • Qt, Qt Mobility Multimedia, Qt GStreamer
slide-6
SLIDE 6
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Development

  • Hardware Adaptation
  • Core OS Platform
  • Applications
slide-7
SLIDE 7
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Development – HW Adaptation

  • Linux Kernel
  • Drivers
  • user space libraries (alsa, v4l2)
  • Middleware plugins
  • GStreamer plugins (Codecs, Camera, Video)
slide-8
SLIDE 8
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Development - Platform

  • libraries from the Linux eco system
  • GStreamer, GUPnP, PulseAudio, Xiph codecs, …
slide-9
SLIDE 9
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Development - Applications

  • QT interfaces are the official one
  • Lower level interfaces might have less stability

guarantees

slide-10
SLIDE 10
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

GStreamer – Use Cases

  • Media Playback
  • Voice and Video Call
  • Camera Still and Video Capture
  • Video Editing
  • Media Transcoding
  • Metadata Indexing and Thumbnailing
slide-11
SLIDE 11
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

GStreamer - Framework

slide-12
SLIDE 12
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Media Playback

  • MeeGo OS:
  • GStreamer playbin2
  • GStreamer components for streaming, parsing,

demuxing and rendering

  • Vendor specific:
  • Hw accelerated codecs
slide-13
SLIDE 13
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

VoIP / Video Call

  • MeeGo OS:
  • Telepathy
  • Lots of GStreamer components
  • Vendor specific:
  • Accelerated voice and video codecs
slide-14
SLIDE 14
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Camera Still and Video capture

  • MeeGo OS:
  • GStreamer camerabin
  • GStreamer elements for format conversion, metadata

(XMP, EXIF), muxing, data routing

  • Vendor specific:
  • Camera source :/
  • Accelerated video and image encoders
slide-15
SLIDE 15
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Video Editing

  • MeeGo OS:
  • GStreamer gnonlin
  • GStreamer components for muxing, demuxing,

format conversion, …

  • Vendor Specific
  • Accelerated codecs
slide-16
SLIDE 16
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Metadata Indexing & Thumbnailing

  • MeeGo Core OS:
  • Tagreadbin (experimental)
  • GStreamer components for parsing, demuxing
  • Vendor specific:
  • nothing
slide-17
SLIDE 17
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Hot Topics

  • s/XVideoOverlays/TextureStreaming/ ?
  • Bitstream passthrough
  • Camera APIs
  • Performance optimizations
  • Accelerated Codecs & Filters
  • Easy to use high level APIs
slide-18
SLIDE 18
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Hot Topic – Video Rendering

  • Xvideo interface is getting old
  • Inherent memcpy
  • Hard to integrate with alpha in Uis
  • Classic GL is not well suited either
  • Texture upload
  • Texture streaming needed
slide-19
SLIDE 19
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Hot Topic – Bitstream passthrough

  • GStreamer normally renders RAW A/V locally
  • Remote devices can also decode
  • BT Headsets, HDMI TV, SPDIF → Receiver
  • Initial support in PulseAudio from Intel
  • GStreamers playbin2 needs work
slide-20
SLIDE 20
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Hot Topic – Camera APIs

  • Phone cameras != USB webcams
  • V4l2 needs to abstract features better
  • Lots of work on GStreamers camerabin(2)
slide-21
SLIDE 21
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Hot Topic - Performance

  • better performance = longer use time
  • camera recording, music/video playback, call
  • locking is slow on ARM
  • overhead for audio part in calls
  • low memory footprint = more stuff cached
slide-22
SLIDE 22
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Hot Topic – Accelerated Plugins

  • Software Codecs and Filters
  • GStreamer uses ORC to enable CPU SIMD use
  • OpenCL would be interesting for GPU use
slide-23
SLIDE 23
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Hot Topic – Hight Level API

  • Qt Mobility Multimedia
  • Mediaplayer API
  • Audiocapture API
  • Camera API
  • More feedback needed

http://doc.qt.nokia.com/qtmobility-1.0/multimedia.html

slide-24
SLIDE 24
  • Dr. Stefan Kost – Nokia - The MeeGo Multimedia Stack - CELF Embedded Linux Conference Europe

Thanks ! Any Questions ?