P2P Audio Streaming for the iPod Touch P2P Audio Streaming for the - - PowerPoint PPT Presentation

▶
p2p audio streaming for the ipod touch p2p audio
SMART_READER_LITE
LIVE PREVIEW

P2P Audio Streaming for the iPod Touch P2P Audio Streaming for the - - PowerPoint PPT Presentation

Nov 13, 2022 194 likes •719 views

P2P Audio Streaming for the iPod Touch P2P Audio Streaming for the iPod Touch Student: Tran, Thanh Minh Tu Student: Tran, Thanh Minh Tu Advisor: Prof. Dr. Thomas Schmidt Advisor: Prof. Dr. Thomas Schmidt Information Engineering Information

slide-1
SLIDE 1

P2P Audio Streaming for the iPod Touch P2P Audio Streaming for the iPod Touch Student: Tran, Thanh Minh Tu Student: Tran, Thanh Minh Tu Advisor: Prof. Dr. Thomas Schmidt Advisor: Prof. Dr. Thomas Schmidt Information Engineering Information Engineering HAW Hamburg – iNet Lab HAW Hamburg – iNet Lab iPhone University Program iPhone University Program Hamburg - 19.11.2009 Hamburg - 19.11.2009

slide-2
SLIDE 2

Overview

 Introduction  Overlay P2P Network  Application Layer Multicast (ALM)  Audio Processing on the iPhone  P2P Audio Streaming for the iPod Touch

(PAS4i)

 Discussion

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-3
SLIDE 3

Introduction

 The P2P file sharing systems  The P2P media streaming systems  Our story  The PAS4i application

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-4
SLIDE 4

The P2P file sharing systems

 Napster (1999)  Gnutella  BitTorrent

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-5
SLIDE 5

The P2P media streaming systems

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-6
SLIDE 6

The P2P media streaming systems

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-7
SLIDE 7

Our story

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-8
SLIDE 8

The PAS4i application

 PAS4i: “P2P Audio Streaming for the iPod

Touch”

Introduction Overlay P2P ALM iPhone PAS4i Discussion

9

±

¯ ¯ ¯ ¯ ¯ ¯

slide-9
SLIDE 9

Client-Server model

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Client-Server For large scale system: High deployment cost!!!

slide-10
SLIDE 10

Overlay P2P Network

 Unstructured P2P

 Centralized P2P  Pure P2P  Hybrid P2P

 Structured P2P

 DHT-based: Chord, Pastry, CAN etc.

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-11
SLIDE 11

Unstructured P2P

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Pure P2P Centralized P2P Hybrid P2P Central entity (indexing) provides the service No central entity Any peer can be removed Dynamic central entities Any peer can be removed Central server: single point of failure Flooding: communication overhead Pros and Cons solution

  • f both Centralized &

Pure P2P

slide-12
SLIDE 12

Unstructured P2P

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Pure P2P Centralized P2P Structured P2P

slide-13
SLIDE 13

Structured DHT-based P2P

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Hi, I am looking for BabyOneMoreTime.mp3 I have or I know where BabyOneMoreTime.mp3 is located.

Node locations on the underlay IP network

slide-14
SLIDE 14

Structured DHT-based P2P

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Node locations on the Structured P2P network

IP: stanford.edu Key: 009 Routing table: 120, 280

P2P user: search for BabyOneMoreTime.mp3 Song key = 810

IP: 85.8.10.30 Key: 280 Routing table: 400, 545 IP: 7.31.10.25 Key: 545 Routing table: 691 IP: haw-hamburg.de Key: 691 Routing table: 545, 870

Key: 120

Key: 400 Key: 870

BabyOneMoreTime.mp3 Key: 810 Step 1: forwarding Step 2: forwarding Step 3: forwarding Step 4: responding

slide-15
SLIDE 15

Structured DHT-based P2P – Pastry approach

 Routing information of a Pastry node

Introduction Overlay P2P ALM iPhone PAS4i Discussion

key: 103200 key: 102022

slide-16
SLIDE 16

Structured DHT-based P2P – Pastry approach

 Pastry API

 pastryInit(Credentials, Application)  route(msg, key)  send(msg, IP-addr)  deliver(msg, key)  forward(msg, key, nextId)  newLeafs(leafSet)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-17
SLIDE 17

Structured DHT-based P2P

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-18
SLIDE 18

Application Layer Multicast (ALM)

 IP Multicast vs. ALM  ALM characteristics  ALM media Live streaming topologies

 Tree-first vs. Mesh-first  Scribe – a Tree-first approach

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-19
SLIDE 19

IP Multicast vs. ALM

Introduction Overlay P2P ALM iPhone PAS4i Discussion

IP Multicast ALM

slide-20
SLIDE 20

ALM characteristics

 Deployment level (i.e., Proxy-based or End-

system ALM)

 Application domain (e.g., number of user,

latency constraint, network resource etc.)

 Multicast management approach  Routing mechanism

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-21
SLIDE 21

Streaming topologies – Tree-first

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Single-tree approach Multi-tree approach No upload bandwidth for leaf nodes All nodes contribute the upload bandwidth

slide-22
SLIDE 22

Streaming topologies – Tree-first

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Peer Churn Robustness in Tree-based approach?

slide-23
SLIDE 23

Scribe – a Tree-based approach

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Streaming tree: Scribe Built on the Pastry Overlay Pastry Overlay Network

slide-24
SLIDE 24

Scribe – a Tree-based approach

Introduction Overlay P2P ALM iPhone PAS4i Discussion

 Scribe API

 create(credentials, groupId)  join(credentials, groupId, msgHandler)  leave(credentials, groupId)  multicast(credentials, groupId, msg)

slide-25
SLIDE 25

Scribe – group creation

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-26
SLIDE 26

Scribe – group joining

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-27
SLIDE 27

Scribe – multicast tree

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-28
SLIDE 28

Scribe – group multicasting

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-29
SLIDE 29

Scribe – multicast tree repairing

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-30
SLIDE 30

Streaming topologies – Mesh-first

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Poor mesh topology Streaming tree is built On improved mesh Communication overhead?

slide-31
SLIDE 31

Audio Processing on the iPhone

 Layers of iPhone OS  The three API layers of Core Audio

 Audio Queue Services  Audio File Stream Services  Audio Unit Services (Converter unit, Multichannel

Mixer unit, remoteIO unit)

 Audio Processing Graph Services  etc. (not used in Pas4i)

 Audio Codec

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-32
SLIDE 32

Layers of iPhone OS

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Graphics Technologies Audio Technologies Three API Layers Video Technologies

slide-33
SLIDE 33

The three API layers of Core Audio

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Only available for Mac OS X (not in iPhone OS) Used by PAS4i

slide-34
SLIDE 34

Audio File Stream Services (1st)

 To play streamed audio content (i.e., from

network connection)

 Used with Audio Queue Services  Parse audio packets and metadata in a

network bitstream or on-disk files (two callback

functions for audio packet and metadata)

 Supported formats: MP3, AAC, WAVE,

MPEG-4 etc.

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-35
SLIDE 35

Audio Queue Services (AQS) (2nd)

 Provides low overhead way to record and

play audio

 AQS and AVAudioPlayer class: the only ways

to play compressed audio (e.g., MP3)

 Uses with Audio File Services or Audio File

Stream Services

 Audio Queue Callback Functions for

Recording and Playback

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-36
SLIDE 36

Playback with AQS (2nd)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-37
SLIDE 37

Recording with AQS (2nd)

 For recording, the audio callback has a

converse role

 Not used in the PAS4i: WHY?

 recording callback occurred only every 16384

samples (latency of 0.37s for 44.1 kHz)

 unacceptable for real-time audio processing  => have to use Audio Unit remoteIO (callback

  • ccurred for every 1024 samples)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-38
SLIDE 38

Audio Unit Services (AU) (3rd)

 Provides lowest latency audio, especially when

doing simultaneous input and output e.g., VoIP

 Converter unit  Multichannel mixer unit  RemoteIO unit  etc. (not used in PAS4i)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-39
SLIDE 39

AU Converter (3rd)

 Provides sample rate, bit depth, and bit

format (linear to fixed-point) conversions

 Conversion is done only between PCM (Audio

Converter Services for conversion with other compressed formats)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

AU Converter I n p u t

O u t p u t

input bus 0

  • utput bus 0

PCM PCM*

slide-40
SLIDE 40

AU Multichannel mixer (3rd)

 Used to mix multiple audio streams to a

single stream

 Only works with PCM

Introduction Overlay P2P ALM iPhone PAS4i Discussion

AU Multichannel mixer

I n p u t

O u t p u t

PCM input bus 0

  • utput bus 0

PCM Mixed stream PCM input bus 1 PCM input bus n

slide-41
SLIDE 41

AU remoteIO (3rd)

 Allows to connect to inputs (from microphone

  • r from application data) and outputs (to

speaker or to application)

 Only works with PCM

Introduction Overlay P2P ALM iPhone PAS4i Discussion

AU remoteIO I n p u t

O u t p u t

input bus 0 – from app. input bus 1 – from mic.

  • utput bus 0 – to speaker
  • utput bus 1 – to app.
slide-42
SLIDE 42

Audio Processing Graph Services (4th)

 Provides interfaces for representing a set of

audio units, connections between their inputs and outputs, and callbacks used to provide inputs

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-43
SLIDE 43

Audio Processing Graph Services (4th)

 Graph connection for audio processing

Introduction Overlay P2P ALM iPhone PAS4i Discussion

AU Multichannel mixer AU Converter AU remoteIO

¯¯¯

bus 0 – from app. bus 0 – to speaker

±

bus 1 – from mic. bus 1 for voice bus 0 for music Callback bus 1 – to app.

slide-44
SLIDE 44

iPhone audio Codec

 Hardware-assisted codec

 For decoding MP3, ACC, ALAC

 Offline rendering: AudioQueueOfflineRender  High performance: fast and save battery  Only play a single format

 Software codec (Audio Converter Services)

 Support a wide range of audio data format  Support simultaneous playback of multiple sounds

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-45
SLIDE 45

PAS4i - Layers

Introduction Overlay P2P ALM iPhone PAS4i Discussion

Node locations on the underlay IP network Node locations on the overlay P2P network Node locations on a single-tree streaming topology Sender Rendezvous Point Rendezvous Point RP RP 1 2 6 10 5 4 3 7 8 9 Sender

GUI & App. Logic & Audio Processing Scribe: multicast tree Pastry: overlay network

slide-46
SLIDE 46

PAS4i - Components

9

±

¯ ¯ ¯ ¯ ¯ ¯

 oRTP library for RTP and RTCP  Chimera – KBR service (similar to Pastry)  Scribe – Multicast service  PAS4i – GUI, audio processing (using iPhone

Core Audio Unit)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-47
SLIDE 47

PAS4i - Functionalities

9

±

¯ ¯ ¯ ¯ ¯ ¯

 Routing service (Pastry)  Join the Pastry overlay (Pastry)  Create a streaming group (Scribe)  Join a group (Scribe)  Leave a group (Scribe)  Send multicast karaoke/music stream to a group (Scribe)  Tree maintenance (Scribe – not yet)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-48
SLIDE 48

PAS4i - Usage

9

±

¯ ¯ ¯ ¯ ¯ ¯

 Send multicast or receive MP3 music stream (Audio

File Stream Services + Audio Queue Services)

 Send multicast or receive karaoke background

music stream for low PCM (Audio Unit Services: remoteIO,

converter)

 Sing karaoke with local or with received background

music stream and multicast to another group (Audio

Processing Graph Services + Audio Unit Services: remoteIO, converter, multichannel mixer)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-49
SLIDE 49

PAS4i – Future Implementation

9

±

¯ ¯ ¯ ¯ ¯ ¯

 Tree maintenance function  Using hardware-assisted and software audio

codecs for streaming efficiency

 Voice chat for group communication  Adding karaoke lyric and meta data (song name,

singer, location etc.) together with the music stream

 Feedback from listeners (grading, etc.)

Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-50
SLIDE 50

Discussion

 Live streaming vs. Video-on-demand streaming  Scribe ALM via KBR conflicts heterogeneity issue for

streaming systems

 Non-DHT, key generation takes heterogeneity and locality

  • f nodes in consideration

 Considerations: peer churn, communication overhead, NATs

and Firewalls, network coding

 ISP friendly systems  IP Multicast in combination with ALM?  ALM needs more study as well as ...  PAS4i needs improvement Introduction Overlay P2P ALM iPhone PAS4i Discussion

slide-51
SLIDE 51

Questions

Introduction Overlay P2P ALM iPhone PAS4i Discussion