Practical Experiences from Using Pulseaudio in Embedded Handheld - - PowerPoint PPT Presentation

practical experiences from using pulseaudio in embedded
SMART_READER_LITE
LIVE PREVIEW

Practical Experiences from Using Pulseaudio in Embedded Handheld - - PowerPoint PPT Presentation

Practical Experiences from Using Pulseaudio in Embedded Handheld Device Jyri Sarha 18.9.2009 Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 2 Background Information Name: Jyri Sarha Education: Master


slide-1
SLIDE 1

Practical Experiences from Using Pulseaudio in Embedded Handheld Device

Jyri Sarha 18.9.2009

slide-2
SLIDE 2

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 2

Background Information

  • Name: Jyri Sarha
  • Education: Master of Computer Science, Helsinki University of

Technology 2000

  • Employer: Nokia Devices / Maemo
  • Responsibility: Audio Subsystem Architecture and

Development

slide-3
SLIDE 3

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 3

Audio Features N900 Linux Phone

  • Runs Pulseaudio
  • Primary audio API is libpulse0

Used mostly through gstreamer and libcanberra

  • Pulseuadio applies transducer specic audio processing and

speech pre- and post-processing

  • Implements integrated VoIP and Cellular call functionality

All audio is played through Pulseaudio IOW the phone has APE Centric Audio Architecture

slide-4
SLIDE 4

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 4

Typical Smart Phone Audio Architecture

  • Cellular modem has direct connection to audio HW

Cellular call is power ecient and easy to implement

  • Application Engine routes audio through cellular modem

Cellular modem wastes power in music playback use case

  • Add alternative audio route from APE to audio HW

HW design is complex and expensive Audio routing, mixing and processing becomes complicated

slide-5
SLIDE 5

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 5

What is APE Centric Audio Architecture

  • Audio HW connected directly Application Engine
  • Cellular call audio is routed via APE like any other application
  • Cellular modem becomes more like data modem
slide-6
SLIDE 6

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 6

Advantages & Challenges of APE Centric AA

  • Advantages

All audio routing and mixing can be done in APE Similar architecture to PC or laptop VoIP and CS-call can share same audio processing pipeline Conclusion: Simplies SW Fewer functional requirements for Cellular modem Less wiring on HW layout Conclusion: Simplies HW

  • Challenges

Cellular call latency CS-call audio processing consumes APE CPU Both Cellular Modem and APE cosume power during call

slide-7
SLIDE 7

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 7

N900 Audio HW

Simplied picture of N900 audio HW

slide-8
SLIDE 8

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 8

Implementation Challenges

  • Minimize cellular call latency increase caused by the

architecture

  • Audio pipeline optimization to minimize power and CPU

consumption

  • Accurate feedback loop timing for Acoustic Echo Cancellation
slide-9
SLIDE 9

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 9

Cellular Call Latency Challenge

  • Cellular modem air interface alone causes a lot of latency
  • GSM and 3G audio frame size is 20 ms
  • Cellular frame timing is ruled by base station
  • Timing of 20ms frames change in cellular hand over
  • Simple buering between cellular modem and audio codec adds

at least 20 ms latency to each direction

slide-10
SLIDE 10

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 10

Minimize Cellular Call Latency

  • Run ALSA with two 5 ms fragments

This is doable even on ARM Linux with real-time priority DMA buering delay is 5ms each direction

  • Synchronize up link audio buering with Cellular Modem

Cellular Modem sends up-link timing adjustment messages Align up-link buering according to messages Change UL timing with 5 ms granularity

  • Synchronize down link audio buering with Cellular Modem

Keep tight buer management to minimize latency

slide-11
SLIDE 11

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 11

Use Time Optimization

  • Do ecient power management
  • Optimize CPU usage

Use NEON vectorization when applicable Optimize all audio processing for APE CPU Including: Speex SRC and Nokia proprietary algorithms

slide-12
SLIDE 12

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 12

Ecient Power Management

  • Turn all possible power domains o as often as possible
  • Take advantage of McBSP2 (Multi channel Buered Serial

Port) Do block transfers to McBSP2 1280 word buer Power whole CPU down between blocks including DMA Use bigger fragments when not in CS-call

slide-13
SLIDE 13

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 13

Acoustic Echo Cancellation the Basic Idea

  • Correlate echo reference from the mic input to nd alignment
  • Filter echo reference out from the mic input
  • Good acoustic echo path modeling for transient signals is

possible only with proper time alignment of the reference loop

slide-14
SLIDE 14

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 14

Accurate feedback loop timing for AEC

  • Accurate playback and capture latencies are needed for time

alignment of echo reference and mic signal

  • Latency functions of ALSA do not work well with DMA doing

block transfers

  • Solution: Implement ALSA sink latency functions based on

snd_pcm_htimestamp()

slide-15
SLIDE 15

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 15

N900 Pulseaudio Conguration

slide-16
SLIDE 16

Practical experiences from using Pulseaudio in embedded handheld device, 18.9.2009 16

Thanks!

  • to Lennart Poettering for the invitation
  • to Kai Vehmanen, Kalervo Kontola and Seppo Ingalsuo for

ideas and support

  • to Marc-Andr Lureau for encouragement to come