CN Computer Networks Research Group G R Department of Computer - - PowerPoint PPT Presentation

Empirical Evaluation of Upstream Throughput in a DOCSIS Access Network

Swapnil Bhatia (with Radim Bartoˇ s and Chaitanya Godsay) Computer Networks Research Group Department of Computer Science

CN

R

G

and The InterOperability Laboratory Research Computing Center University of New Hampshire Durham, NH 03824

✬ ✫ ✩ ✪

Objectives of this Talk

◮ Report measurement results from our DOCSIS testbed ◮ Describe our approach to interpreting results ◮ Promote discussion of practical aspects of access networks ◮ Solicit feedback and ideas from the audience about each of the above ◮ Promote further collaborative study of access networks

2 of 30 MSAN 2005

✬ ✫ ✩ ✪

Outline

◮ Introduction ⋄ DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). ◮ Background of this study ⋄ InterOperability Lab., vendors and providers, complexity of standard. ◮ Overview of Experiments ⋄ Testbed, variables and data interpretation. ◮ Results ⋄ Subset of conclusions. ◮ Summary and discussion

3 of 30 MSAN 2005

✬ ✫ ✩ ✪

DOCSIS Introduction

(Source: DOCSIS 1.1 RFI Specification)

◮ DOCSIS — Data Over Cable Service Interface Specification ⋄ MAC protocol utilizing existing CATV network ⋄ Developed by CableLabs (Louisville, CO) ⋄ Version 1.0 (pre-1999), 1.1, (1999-), 2.0 (2004-05)

4 of 30 MSAN 2005

✬ ✫ ✩ ✪

DOCSIS Introduction (contd.)

◮ Tree topology ◮ Downstream vs. upstream ⋄ Separate frequencies ⋄ Broadcast, unicast (resp.) ⋄ TDMA upstream

CM 2 CM 1

User 1 data User 2 data

CM n

User n data

CM 3

User 3 data CATV Plant

CMTS

CM − Cable Modem CMTS − CM Termination System Splitter/Combiner WAN To

◮ MAP: Periodic downstream control message ⋄ Describes upstream transmission schedule ⋄ Who: Which CM transmits? ⋄ When: Starting when and how long? ⋄ What: What can it transmit? ◮ Different types of transmission windows ⋄ BW Request (BWR), BWR or Data, Short Data, Long Data, Maintenance.

5 of 30 MSAN 2005

✬ ✫ ✩ ✪

DOCSIS Introduction (contd.)

(Source: DOCSIS 1.1 RFI Specification) 6 of 30 MSAN 2005

✬ ✫ ✩ ✪

DOCSIS Introduction (contd.)

(Source: DOCSIS 1.1 RFI Specification)

◮ Basic Data Transmission Cycle ⋄ Wait for contention-based BW Request window ⋄ Send Request (with retries) ⋄ Retry until MAP received ⋄ Wait for start of MAPped window ⋄ Send data ◮ Alternatives ⋄ Unicast data or request windows

7 of 30 MSAN 2005

✬ ✫ ✩ ✪

DOCSIS Introduction (contd.)

Performance Enhancers ◮ Piggybacking ⋄ Use part of data transmission window to make new requests ◮ Concatenation ⋄ Transmit more than one data PDU in a single transmission window ◮ Fragmentation ⋄ Divide large data PDU to fit into current transmission window ◮ Header Suppression ⋄ Header of data PDU suppressed at CM, regenerated at CMTS

8 of 30 MSAN 2005

✬ ✫ ✩ ✪

DOCSIS Introduction (contd.)

Performance Enhancers ◮ Piggybacking ⋄ Use part of data transmission window to make new requests ◮ Concatenation ⋄ Transmit more than one data PDU in a single transmission window ◮ Fragmentation ⋄ Divide large data PDU to fit into current transmission window ◮ Header Suppression ⋄ Header of data PDU suppressed at CM, regenerated at CMTS

9 of 30 MSAN 2005

✬ ✫ ✩ ✪

Outline

◮ Introduction ⋄ DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). ◮ Background of this study ⋄ InterOperability Lab., CableLabs, complexity of standard. ◮ Overview of Experiments ⋄ Testbed, variables and data interpretation. ◮ Results ⋄ Subset of conclusions. ◮ Summary and discussion

10 of 30 MSAN 2005

✬ ✫ ✩ ✪

Background of this Study

◮ Supported by the UNH InterOperability Laboratory ⋄ Largest standards compliance testing facility in the country ⋄ 19 consortia (including: iSCSI, SATA, IPv6, WiMax, EFM . . . ) ⋄ Industry supported, driven testing and applied research ◮ Conformance, interoperability and performance ⋄ Previously verified, but in isolation ◮ Bottomline for vendors and service providers ⋄ Configuration design ⋄ Measurements with real devices ◮ Benefits to ⋄ Protocol designers ⋄ Equipment manufacturers ⋄ Service providers

11 of 30 MSAN 2005

✬ ✫ ✩ ✪

Outline

◮ Introduction ⋄ DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). ◮ Background of this study ⋄ InterOperability Lab., CableLabs, complexity of standard. ◮ Overview of Experiments ⋄ Testbed, variables and data interpretation. ◮ Results ⋄ Subset of conclusions. ◮ Summary and discussion

12 of 30 MSAN 2005

✬ ✫ ✩ ✪

Overview of Experiments

◮ Goal ⋄ Characterize upstream performance to answer deployment design questions of the type: ⋄ When is it better to piggyback than concatenate? ⋄ How much is the improvement using concatenation? ⋄ Dependent or independent of CMTS scheduling algorithm ◮ Independent variables ⋄ Upstream channel rate ⋄ Input packet length ⋄ Performance enhancer ⋄ CMTS ◮ Dependent variables ⋄ Throughput ⋄ Latency

Traffic generator and analyzer Coaxial cable Ethernet CMTS Upstream data RF analyzer CM CM CM

13 of 30 MSAN 2005

✬ ✫ ✩ ✪

Overview of Experiments

◮ Independent variables ◮ Upstream channel rate ⋄ {0.64, 1.28, 2.56, 5.12, 10.24} Mpbs. ◮ Packet length ⋄ {64, 128, 256, 512, 768, 1262, 1500} bytes. ◮ Performance enhancer ⋄ {Concatenation, Piggybacking, Both, Neither} allowed. ◮ CMTS ⋄ {Vendor-A, Vendor-B}. ◮ Load ⋄ Constant load of 8 Mbps (saturation).

14 of 30 MSAN 2005

✬ ✫ ✩ ✪

Overview of Experiments

◮ Define a configuration as a tuple ⋄ < rate, length, enhancer, cmts > ◮ Define a transition as a doubleton of configurations ⋄ {< v1, v2, v3, v4 >, < u1, u2, u3, u4 >} such that ∃! (1 ≤ i ≤ 4) (vi = ui) ◮ Consider a k−tuple of n1, . . . , nk -valued attributes each ◮ Total number of transitions is N =

k

• i=1

ni(ni − 1) 2 ·

• j=i

nj

• =

k

• i=1

ni ·

k

• i=1

(ni − 1) 2

15 of 30 MSAN 2005

✬ ✫ ✩ ✪

Overview of Experiments

◮ 2400 cases ⋄ An experiment for each transition ⋄ Capture effect of a single change ⋄ 25 runs per experiment ◮ Decide whether change improves or worsens performance ⋄ Statistically robust, unbiased data interpretation ⋄ Between and across CMTS

16 of 30 MSAN 2005

✬ ✫ ✩ ✪

Overview of Experiments

◮ Wilcoxon Signed Rank Sum Test (WSRS) ⋄ A popular hypothesis test independent of distribution of data ⋄ Calculates probability of median of sorted ranks being zero ⋄ Null hypothesis (NH): no change in throughput due to a transition (Toriginal − Tchanged = 0) ⋄ Test provides probability P of NH being true ⋄ Fix desired significance level α = 0.05 ⋄ If P ≤ α, reject NH (Toriginal − Tchanged = 0) ⋄ i.e., transition affects throughput ⋄ Check one-sided alternative (Toriginal − Tchanged > 0, Toriginal − Tchanged < 0?) ◮ Actual α = 0.05/2400 (Bonferroni correction)

17 of 30 MSAN 2005

✬ ✫ ✩ ✪

Outline

◮ Introduction ⋄ DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). ◮ Background of this study ⋄ InterOperability Lab., CableLabs, complexity of standard. ◮ Overview of Experiments ⋄ Testbed, variables and data interpretation. ◮ Results ⋄ Subset of conclusions. ◮ Summary and discussion

18 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

0.5 1 1.5 2 2.5 3 200 400 600 800 1000 1200 1400 1600 Throughput (Mbps) Packet length (bytes) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps

(a) No enhancers

0.5 1 1.5 2 2.5 3 200 400 600 800 1000 1200 1400 1600 Throughput (Mbps) Packet length (bytes) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps

(b) Both enhancers ◮ Per CMTS (99% confidence) ⋄ Maximum throughput < 3 Mbps per CM ⋄ Enhancers effective for smaller packets

19 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

0.5 1 1.5 2 2.5 3 200 400 600 800 1000 1200 1400 1600 Throughput (Mbps) Packet length (bytes) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps

(c) Concatenation

0.5 1 1.5 2 2.5 3 200 400 600 800 1000 1200 1400 1600 Throughput (Mbps) Packet length (bytes) Channel 0.64Mbps 1.28Mbps 2.56Mbps 5.12Mbps 10.24Mbps

(d) Piggybacking ◮ Per CMTS (99% confidence) ⋄ Concatenation very effective for smaller packets ⋄ Piggybacking largely ineffective ⋄ Need more CMs to see effect

20 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

◮ When is Piggybacking useful?

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) No enhancers on 1.28Mbps Piggybacking on 1.28Mbps 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) No enhancers on 2.56Mbps Piggybacking on 2.56Mbps No enhancers on 5.12Mbps Piggybacking on 5.12Mbps

⋄ Larger packet lengths at 1.28 Mbps ⋄ Fewer request windows due to large packets

21 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

◮ Is Piggybacking Ever Better than Concatenation?

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) Concatenation on 0.64Mbps Piggybacking on 0.64Mbps 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) Concatenation on 1.28Mbps Piggybacking on 1.28Mbps

⋄ Yes. ⋄ With large packets on small channels

22 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

◮ Is having both enhancers always useful? ⋄ No. ⋄ With small packets it is detrimental.

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) Concatenation on 0.64Mbps Both Enhancers on 0.64Mbps 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) Concatenation on 1.28Mbps Both Enhancers on 1.28Mbps 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) Concatenation on 2.56Mbps Both Enhancers on 2.56Mbps 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) Concatenation on 5.12Mbps Both Enhancers on 5.12Mbps

23 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

◮ Will an increase in channel rate always help? ⋄ No, with small packets an increase can be detrimental.

0.2 0.4 0.6 0.8 1 1 2 3 Throughput (normalized) Enhancer combination 5.12 Mbps channel 10.24 Mbps channel

24 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

◮ Suppose small packets on a small channel ⋄ What is the most economical way to increase throughput? ⋄ Enable concatenation alone. ◮ Mid-sized packets? ⋄ Must increase channel rate. ◮ Large packets? ⋄ Enable piggybacking, or increase channel rate

25 of 30 MSAN 2005

✬ ✫ ✩ ✪

Results

◮ Do both CMTS agree on all responses? ⋄ No.

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) None on 0.64 Mbps None on 1.28 Mbps 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 200 400 600 800 1000 1200 1400 Throughput (normalized) Packet Length (bytes) None on 1.28 Mbps None on 0.64 Mbps

◮ Anomalies excluded from results ⋄ Useful to respective CMTS vendors

26 of 30 MSAN 2005

✬ ✫ ✩ ✪

Outline

◮ Introduction ⋄ DOCSIS architecture, protocol, enhancers (piggybacking, concatenation, fragmentation etc.). ◮ Background of this study ⋄ InterOperability Lab., CableLabs, complexity of standard. ◮ Overview of Experiments ⋄ Testbed, variables and data interpretation. ◮ Results ⋄ Subset of conclusions. ◮ Summary and Discussion

27 of 30 MSAN 2005

✬ ✫ ✩ ✪

Summary

◮ Characterized upstream performance of a DOCSIS system ⋄ Channel rate, packet sizes, enhancers and CMTS ⋄ Presented only a subset of results ◮ Empirical Results ⋄ Exhaustive, measurement-based, real system-level ⋄ Valuable tool for configuration design ⋄ Black box approach ◮ Data currently being analyzed by vendors ⋄ Also available at http://www.cs.unh.edu/cnrg/docsis ◮ Future work ⋄ Multi-CM characterization ⋄ Other QoS enhancers ⋄ Comparison with analytical models

28 of 30 MSAN 2005

✬ ✫ ✩ ✪

Future Work

◮ Classification and Regression Tree Model

29 of 30 MSAN 2005

✬ ✫ ✩ ✪

Discussion and Questions

30 of 30 MSAN 2005