Emulation of Ad Hoc Networks David A. Maltz, Qifa Ke, David B. Johnson CMU Monarch Project Computer Science Department Carnegie Mellon University http://www.monarch.cs.cmu.edu/ Carnegie Mellon CMU Monarch Project monarch@monarch.cs.cmu.edu
Early Emulation Successes Stress testing of networking code via macfilter � Used to debug our 8 node ad hoc network testbed � Description of interesting/problematic movement scenarios created � Changing network topology extracted into trace file � Physical machines running actual implementation synchronously read trace-file � Machines prevented from receiving packets from non-neighbors � Described at previous IETF and in CMU CS Tech Report 99-116 Found to be critical for achieving code stability, however cannot be used for performance evaluation � Propagation model grossly simplified � MAC-layer effects unaccounted for CMU Monarch Project monarch@monarch.cs.cmu.edu
Goal of Current Emulation Work Evaluate real systems under ad hoc network conditions: � Performance study of network protocols or applications � Without implementing them in ns-2 � Without deploying and operating physical machines in the field � Using real implementation and real user pattern � Only the network environment is simulated Evaluating real systems with just simulation is not practical: � Difficult to model real applications inside the simulator � Real systems are significantly performance tuned Example: Evaluate performance of CMU Coda file system in an ad hoc network: � Coda is 100+ KLOC implemented and tuned � Coda researchers have expertise to evaluate their system but can’t create network environments CMU Monarch Project monarch@monarch.cs.cmu.edu
Method � Leverage emulation work by Kevin Fall and VINT project � ns-server is a single central machine running ns-2 � Real application code runs on real machines directing real packets to “default router” (ns-server) � User creates scenarios to control the environment inside ns-2 : – Movement pattern of all mobile nodes inside ns-2 – Background traffic MN1 MN2 ns-server MN4 MN3 Logical view of emulation setup CMU Monarch Project monarch@monarch.cs.cmu.edu
Informal Validation of Emulation Method Example: Compare FTP behavior between simulation and emulation: � ns-server: 450 MHz Pentium II CPU, running FreeBSD 3.1 � 10 simulated nodes � Compare 30 MB FTP transfer between simulation and emulation � 2 additional simulated FTP connections for background traffic 7 3.5x 10 3 TCP time–sequence # plots 2.5 bytes transfered have basically the same shape: 2 1.5 1 But how well does this scale? 0.5 simulation emulation 0 0 50 100 150 200 250 300 time (seconds) CMU Monarch Project monarch@monarch.cs.cmu.edu
A More Challenging Example � 16 simulated nodes moving in a complicated pattern with CBR and FTP background traffic � 30 MB FTP transfer simulated in ns-2 � 30 MB FTP transfer between two physical nodes with emulation 7 3x 10 2.5 bytes transfered 2 1.5 1 0.5 simulation emulation 0 0 100 200 300 400 500 time (seconds) CMU Monarch Project monarch@monarch.cs.cmu.edu
Evaluating Emulation Scalability Limitations on emulation scalability: � System effects: scheduling delays, potential buffer overruns � Additional latency of transferring packets to/from ns-server � Only so much one CPU can do Experimental scenario: SINK � Conduct real FTP transfer between 2 physical nodes ��� ��� �� �� ��� ��� �� �� � Vary the number of other �� �� ��� ��� �� �� SRC ���� ���� ���� ���� �� �� � � � � � � simulated nodes in ns-2 � � 30 M Bytes CBR � Vary the amount of background traffic Scalability Test Scenario CMU Monarch Project monarch@monarch.cs.cmu.edu
Scalability Results Summary Ideally: emulation throughput = simulation throughput � Pure simulation achieves 1.49 Mb/s on a 2 Mb/s link � Emulation achieves throughput that depends on overall simulator workload: # of CBR packets/second 0 10 50 100 300 # of nodes 1.44 Mb/s 4 1.30 Mb/s 8 960 Kb/s 16 FTP Throughput Achieved CMU Monarch Project monarch@monarch.cs.cmu.edu
Does the Simulator Keep up with Real Time? � For event scheduled at t 1 but processed at t 2 : time-lag = t 2 – t 1 � Record the time-lag for each delayed event � Histograms show almost all time-lags are less than 4 ms 100 CBR packets/sec 16 nodes 6 10 5 10 # of time-lag events 4 10 3 10 2 10 1 10 4 54 104 154 204 ms Max lag: 355 ms Total # of events: 1.64e+07 CMU Monarch Project monarch@monarch.cs.cmu.edu
All of the Histograms 0 packets/sec 10 packets/sec 50 packets/sec 100 packets/sec 300 packets/sec 4 simulated nodes 6 10 5 10 4 10 3 10 2 10 1 10 1.03e+06 1.30e+06 2.38e+06 3.73e+06 9.13e+06 8 simulated nodes 6 10 5 10 4 10 3 10 2 10 1 10 1.65e+06 2.28e+06 4.80e+06 7.95e+06 2.06e+07 16 simulated nodes 6 10 5 10 4 10 3 10 2 10 1 10 4 54 104 154 204 4 54 104 154 204 4 54 104 154 204 4 54 104 154 204 4 54 104 154 204 2.89e+06 4.24e+06 9.65e+06 1.64e+07 4.35e+07 CMU Monarch Project monarch@monarch.cs.cmu.edu
Conclusion Emulation works for networks of sufficient size and complexity to enable study of interesting application scenarios � Using direct emulation based on ns leverages future improvements to propagation models � The bottleneck is the real-time requirement in ns-server � Currently using to evaluate the Coda distributed file system Open questions: � How far can this technique be generalized? � How to ensure performance measurements are meaningful? – Open loop: evaluate emulation system on many scenarios using reference application with good simulation models – Closed loop: after each emulation run, examine emulation trace to determine if emulation system introduced artifacts – Validate against real system in ad hoc network testbed CMU Monarch Project monarch@monarch.cs.cmu.edu
Recommend
More recommend
