Services in 4G WiMAX Networks Amitabha Ghosh IBM India Research - - PowerPoint PPT Presentation

IBM Research 1

QoE Characterization for Video-On-Demand Services in 4G WiMAX Networks

Amitabha Ghosh IBM India Research Laboratory

Department of Electrical Engineering University of Southern California, Los Angeles http://anrg.usc.edu/~amitabhg amitabhg@usc.edu

IBM Research

Talk Outline

 Motivation  Preliminary Experiments  Survey

 Protocol Overview (RTP)  QoE Metrics

 Simulation / Experiments with Video Traces

 ns2, evalvid

 Future Work & Conclusions

IBM Research

 Active deployment of triple and quadruple services

 (“The Fantastic Four”: broadband internet access, television, and telephone with wireless service provisions).

 Real-time, high-quality video and value-added data services over converged networks (e.g., 4G).  Need to guarantee subscribers’ Quality-of-Experience (QoE) and provide differentiated services in the face of heterogeneous end devices, varying wireless channels, resource constraints, etc.  Lack of QoE orchestration models that map network events (e.g., variation in bandwidth, delay, packet error rates, jitter) to QoE.

Motivation

IBM Research

Preliminary Experiments

Helix Server Windows Media Player

Net Limiter Media Tracker RTP RTCP

Stored video Bandwidth Throttler

Video Characteristics

 Bit-rate: 1.96 Mbps  Duration: 230 sec  File size: 56.7 MB  Media Player initial buffering: 30 sec

IBM Research

Preliminary Experiments

50 100 150 200 250 300 5 10 15 20 25 30 35 40 45

Time (sec) Real-Time Frame Rate (fps)

Startup delay 31 sec

Throttled at 91 sec Plays smoothly for ~ 30 sec more after throttling

<----------->

50 100 150 200 250 300 5 10 15 20 25 30 35

Time (sec) Real-Time Frame Rate (fps) Initial Buffering Delay

Without bandwidth throttling (no Net Limiter) Bandwidth throttled to 1.6 Mbps at 60 sec from the start of the video

Observations

 Smooth video playback for initial buffering time after stalling  Stalls frequently after 91 seconds, mimicking real-time frame rate

IBM Research

Preliminary Experiments

50 100 150 200 250 300 5 10 15 20 25 30 35 40

Time (sec) Real-Time Frame Rate (fps)

Throttled at 1.4 Mbps Throttled at 1.2 Mbps Throttled at 1.0 Mbps Throttled at 0.8 Mbps Throttled at 0.6 Mbps 50 100 150 200 250 300 5 10 15 20 25 30 35 40 45

Time (sec) Real-Time Frame Rate (fps)

Initial buffer = 1 sec Initial buffer = 30 sec Initial buffer = 60 sec

Observations

 Stalls frequently after 91 seconds irrespective of throttled bandwidth  Small bandwidth variation and packet losses cause major degradation in QoE

Variation of real-time frame rate with different initial buffering and throttling bandwidths

IBM Research

Talk Outline

 Motivation  Preliminary Experiments  Survey

 Protocol Overview (RTP, RTCP)  QoE Metrics

 Simulation / Experiments with Video Traces

 ns2, evalvid, VLC, RTP dump

 Future Work  Conclusions

IBM Research 8 Source:

Protocol Stack for Multimedia Services

IBM Research 9

 Provides end-to-end transport services for data with real-time characteristics

• Payload type identification, sequence numbering, time stamping, delivery monitoring

 Does NOT provide timely delivery or other QoS guarantees

• Relies on other protocols like RTCP and lower layers

 Does NOT assume the underlying network is reliable and does NOT deliver PDUs in sequence

• Uses sequence number for reconstructing

 Application level framing

• Headers can be modified and/or added to provide information required by applications

 Profile and Payload Format Specification Document

• Defines a set of payload type codec and their mapping to payload formats
• Defines how a particular payload is fragmented and mapped in RTP packets (RFC 3016 for

MPEG-4)

RTP (Real-Time Transport Protocol)

RTP packet containing the configuration information and a video packet

VS Header VO Header VOL Header Video Packet RTP Header

IBM Research

Incremented by one for each RTP PDU:

• PDU loss detection
• Restore PDU sequence

Sampling instant of first data octet

• multiple PDUs can have same timestamp
• not necessarily monotonic
• used to synchronize different
• media streams

Payload type

RTP Header

• Each source of RTP PDUs; unique random 32-bit ID (SSRC)
• Packets with the same SSRC shares the same timing & sequence

number space so a receiver groups packets by SSRC for playback (used by mixers)

Contributing sources

IBM Research

Attributes of QoE

 Session Quality

 Users’ overall experience

• especially from a connection perspective

 Most affected by

• initial buffering, re-buffering during playback, audio-video synchronization, packet

losses, buffer over/under flow, codec, CPU limitation

 Video Quality

 Frame quality, fidelity/smoothness of motion (fps), stalling

 Audio Quality

 Fidelity and Mono/Stereo

• Users may have different perceptions of what they are seeing based on what they are

hearing

 Different kinds of content need different levels of audio

 Nature of Content

 Contributes to the weight of each factor in shaping QoE

e.g., sports video may require video smoothness over picture clarity; a talking news head may require better audio than picture quality

IBM Research

Video Evaluation Schemes & QoE Metrics

 End System Based

 Primarily developed to evaluate various transcoding schemes  Characterize stream after network transmission is done  Cannot isolate network induced impairments, thus cannot recover; various QoE Metrics

 Objective Metrics – based on mathematical models

 PSNR (Peak Signal to Noise Ratio) – most widely used frame to frame calculation; does not correlate very well with human perception; does not take delay, jitter into account  UQI (Universal Quality Index) – based on structural attributes of objects in the scene; separates comparison of structure, luminance, and contrast  SSIM (Structural Similarity Index) – based on Human Visual System; improvement over UQI; starting to replace PSNR

 Subjective Metrics – based on human perception

 MOS (Mean Opinion Score); VQM (Video Quality Metric); PEVQ (Perceptual Evaluation of Video Quality)

IBM Research

Today’s Solution and What is Lacking?

 Error Concealment

 Intends to conceal the visual effects of packet loss by exploiting temporal or spatial correlation with adjacent data  Picture quality may reduce keeping the number of frames constant

 Frame Skipping

 Does not decode a frame unless all packets are received  Picture quality remains same, but stalls occur

 Bit-rate Capping & Switching, TCP, Feedback-based encoding, etc…

 Streaming at a bit rate matching the capabilities of handset &network  Switch streams between different encoded rates  What we need?  In-network elements that can detect events (variation in channel condition, packet error rates, delay, jitter), infer about the experience (QoE Model) and take preventive action (QoE Orchestration) to maintain video quality

IBM Research

Simulation Setup: ns2 + Evalvid

Video Encoder Video Trace Generator MyTraffic Trace MyUDP MyUDP Sink Source Receiver Network YUV video Sender Tracefile Receiver Tracefile Traffic Tracefile

ns2 environment

MPEG4 video QoE Module Base Station

(BS)

Subscriber Station (SS)

MyTrafficTrace (Application/Traffic/myTrace2)

 Extension of ns2 agent Application/Traffic/Trace  Extracts frame type, frame size, and inter- packet time from traffic trace file  Fragments video frames into smaller segments  Sends the segments to the lower UDP layer at appropriate times

MyUDP (Agent/myUDP)

 Extension of the ns2 agent Agent/UDP  Generates sender trace file  Records timestamp, packet ID, and payload size of each transmitted packet

MyUDPSink (Agent/myUdpSink2)

 Receiving agent for the fragmented video frame packets sent by MyUDP  Records timestamp, packet ID, and payload size

• f each received packet in the receiver trace file

Other Features of Evalvid

 Generate received video in compressed format (MPEG4) from receiver trace file and the original video  Decode compressed video into YUV format  Compute PSNR

IBM Research

Evaluation: Sony Video

100 200 300 400 500 600 2 4 6 8 10 12 14 16

Bandwidth (frame bit rate) of the Video time (sec) bandwidth (Mbits/s)

 -  = 3.9 Mbits/s  = 6.7 Mbits/s  +  = 9.5 Mbits/s

Video Characteristics:

Encoder: MPEG-4 Variable Bit Rate (VBR) Frame Size: CIF 352x288

• No. Frames: 17681

GoP Size: 16 (IPPBPPBPPBPPBPPB) Number of i-frames: 1106 Number of p-frames: 7735 Number of b-frames: 8840 Video duration: 589 sec (~10 min) Mean frame size: 0.2256 Mbits SD frame size: 0.1266 Mbits Mean bandwidth: 6.727 Mbits/s SD bandwidth: 2.8 Mbits/s

100 200 300 400 500 600 2 4 6 8 10 12 14 16

time (sec) bandwidth (Mbits/s) Bandwidth (frame bit rate) of the Video

frame i-frame p-frame b-frame

The I-frames occupy only ~20% of the mean bandwidth. One can assign higher priorities to the I-frames and let those packets pass when the wireless channel is bad.

IBM Research

100 200 300 400 500 600 2 4 6 8 10 12 14 16

time (sec) bandwidth (Mbits)

Source BW and Provisioned BW According to Preset Levels src bw provisioned bw 100 200 300 400 500 600

• 500

500 1000 1500 2000 2500 3000 3500 4000 4500

time (sec) cumulative bandwidth (Mbits)

Cumulative SRC and Provisioned BW for Preset Levels cumulative src bw (a) cumulative provisioned bw (a) buffered data (b)-(a)

Preset BW Provisioning:

Provisioned bandwidth levels are preset at the following values: (µ-2) = 1.10, (µ-) = 3.90, (µ-0.5) = 5.36, µ = 6.72, (µ+0.5) = 8.12, (µ+) = 9.52, (µ+1.5) = 10.92, (µ+2) = 12.32, (µ+2.5) = 13.72, (µ+3) = 15.12 Mbits/s

Strategy: Calculate the average source BW at every 10 sec window (configurable). If the calculated average

BW is greater than the current provisioned BW, then provision a BW that is two levels higher than the current. Else provision a BW that is two levels lower than the current.

What is needed : We need a dynamic provisioning of bandwidth, instead of preset levels, so we can withstand the variation in source BW and never let the buffer go empty or negative. Also the buffer content should be above a minimal threshold and should not store unnecessary data.

100 200 300 400 500 600

• 50

50 100 150 200

time (sec) buffered data (Mbits)

Buffefred Data for Preset Provisioning Strategy % duration of stalls = 120/600 = 0.2 = 20%

IBM Research

Evaluation: Silence of the Lambs Movie

Video Characteristics:

Encoder: MPEG-4 Variable Bit Rate (VBR) Frame Size: CIF 352x288

• No. Frames: 53953

GoP Size: 16

• No. B Frames: 1

Number of i-frames: 3373 Number of p-frames: 23606 Number of b-frames: 26974 Video duration: 30 min (1800 sec) Mean frame size: 0.1227 Mbits Mean bandwidth: 3.74 Mbits/s SD bandwidth: 1.65 Mbits/s

200 400 600 800 1000 1200 1400 1600 1800 2 4 6 8 10 12 14 16

time (sec) bandwidth (Mbits/s)

Bandwidth (frame bit rate) of the SOTL Video

 -  = 2.09 Mbits/s  = 3.74 Mbits/s  +  = 5.39 Mbits/s

IBM Research

200 400 600 800 1000 1200 1400 1600 1800

• 1000

1000 2000 3000 4000 5000 6000 7000

time (sec) cumulative bandwidth (Mbits)

Cumulative SRC and Dynamically Provisioned BW, and Buffered Data cumulative src bw (a) cumulative provisioned bw (b) buffered data (b)-(a) 200 400 600 800 1000 1200 1400 1600 1800

• 100
• 50

50 100 150 200

time (sec) buffered data (Mbits)

Buffered Data for Dynamically Provisioned BW % duration of stalls = 40/1800 = 0.02 = 2%

Dynamic Provisioning of BW

 Bandwidth levels are NOT preset as before  Calculate the cumulative moving average µ(i) and standard deviation (i) of source BW for a given window size and provision a BW according to the following:

BW(i+1) = µ(i) + k.(i) + /w – .[jw.BW(j) – w.i.µ(i)]/w

200 400 600 800 1000 1200 1400 1600 1800 2 4 6 8 10 12 14 16 18

time (sec) bandwidth (Mbits/s)

Dynamic BW Provisioning: w=20, =50 Mbits, =2.0 src bw dynamically provisioned bw

IBM Research

Summary & Future Work

 Ground Work  Survey of VoD literature, protocols, and QoE metrics for VoD services

 Developed simulation framework using ns2, evalvid, etc

QoE Model

 Identifies the time instants at which video gets stalled and correlate them with source BW  Predict QoE (no. of stalls, spread / distribution of stalls, duration of stalls, amount of contingency buffer, etc) for a given time and BW provisioning strategy

 QoE orchestration

 First cut dynamic provisioning of BW based on tracking the source bit rate; this works better than preset level-based provisioning  There exists a trade-off between the number of stalls, the spread of stalls, the average duration of stalls, and the amount of buffer content with BW update frequency

• Average duration of stalls increases as update frequency is decreased
• Amount of buffered content is inversely proportional to the BW update frequency

 Improve upon the dynamic provisioning strategy

 Use of Control Theory

 Multiple subscriber stations  Incorporate

 Wireless channel model  Constraints on BW provisioned between BS-SS

IBM Research

Demo

Video Characteristics

 30 fps VBR  Total # of frames: 401  Video duration: 13 sec  Mean bandwidth: 0.11 Mbits/s  SD bandwidth: 0.02 Mbits/s Video Source Router Receiver

0.11 Mbits/s