Why to Study the Internet? It is there, it is a complex system, ... - - PDF document

1

DIMES

Distributed Internet MEasurement and Simulation

…and what does it have to do with PlanetLab

Yuval Shavitt shavitt@eng.tau.ac.il http://www.netdimes.org

DIMES

Why to Study the Internet?

• It is there, it is a complex system, ...
• Understanding its structure and dynamics

– help current applications (WWW) – help designing future application (file sharing) – help improving routing – predict Internet growth

• DIMES is a project set to study the Internet

structure.

2

DIMES

Distributed System Design: Obtaining the Internet Structure

The Internet as a complex system: static and dynamic analysis

Correlating the Internet with the World: Geography, Economics, Social Sciences

Why DIMES?

• The Internet is an engineered system, so

someone must know how it is built, no?

• NO! It is an uncoordinated interconnection
• f Autonomous Systems (ASes=networks).
• No central database about Internet structure.
• Several projects attempt to reveal the

structure: Skitter, RouteViews, …

3

The Internet Structure

routers

The Internet Structure

The AS graph

4

Revealing the Internet Structure Revealing the Internet Structure

5

Revealing the Internet Structure Revealing the Internet Structure

30 new links

7 new links NO new links

Diminishing return! Diminishing return!

⇓ Deploying more boxes does not pay-off

6

Revealing the Internet Structure

To obtain the ‘horizontal’ links we need strong presence in the edge

What is DIMES?

• Distributed Internet measurement and monitoring

– Based on software agents downloaded by volunteers

• Diminishing return?

– Software agents ⇓ – The cost of the first agent is very high – each additional agent costs almost zero

• Capabilities

– Obtaining Internet maps at all granularity level

• connectivity, delay, loss, bandwidth, jitter, ….

– Tracking the Internet evolution in time – Monitoring the Internet in real time

DIMES

7

Diminishing Return?

• [Chen et al 02], [Bradford et al 01]: when

you combine more and more points of view the return diminishes very fast

• What have they missed?

– The mass of the tail is significant

• No. of views

Diminishing Return?

• [Chen et al 02], [Bradford et al 01]: when

you combine more and more points of view the return diminishes very fast

• What have they missed?

– The mass of the tail is significant

• No. of views

8

Diminish … shminimish How many ASes see an edge?

~9000/6000 are seen

• nly by one

9

Challenges

• It’s a distributed systems:

– Measurement traffic looks malicious

• Flying under the NOC radar screens

(Agents cannot measure too much)

– Optimize the architecture:

• Minimize the number of measurements
• Expedite the discovery rate
• BUT agents are

– Unreliable – Some move around Distributed System

complex system

real world

Agents

• To be able to use agents wisely we need agents

profiles:

– Reliablility

• Daily (seen in 7 of the last 10 days)
• Weekly (seen in 3 of the last 4 weeks)

– Location:

• Static
• Bi-homed: where mostly?
• Mobile: identify home base

– Abilities: what type of measurements can it perform?

Distributed System

complex system

real world

10

• Degree distribution

[Faloutsos99,Lakhina03,Barford01,Chen02]

• Clustering coefficient [Bar04]
• Disassociativity [Vespigni]
• Network motifs (ala Uri Alon)

Distributed System

complex system

real world

Static Internet Graph Analysis Degree Distribution

k

Pr(k) <k>

2 4 6 8 10 12 2 4 6 8 10 12 14 log(degree) log(Pr(degree)) DIMES+BGP (Feb 05) 2 4 6 8 10 12 14 16 2 4 6 8 10 12 log(rank) log(degree) DIMES+BGP (Feb 05)

Zipf plot

11

AS map for July 2005

BGP

• 20585 nodes
• 45720 edges
• <k> = 4.44

DIMES

• 14332 nodes
• 60134 edges
• <k> = 8.39

33,862 edges ⇒DIMES has doubled the connectivity 11,858 edges

AS map for July 2005

BGP

• 20585 nodes
• 45720 edges
• <k> = 4.44

DIMES

• 14332 nodes
• 60134 edges
• <k> = 8.39

33,862 edges 21,538 in both maps 38,596 new edges 11,858 edges +

81,672 edges <k> > 7.80

12

The Internet as a real world mirror

• Changes in the world effect the Internet growth
• To model Internet growth one needs to take into

account

– Geographic location – Political/caltural biases – Economic development – Human rights issues

Distributed System

complex system

real world

13

Internet and the World

• City connectivity map
• Correlation between population*wealth and

Internet size

• Correlation between trade and Internet

connectivity

• PoP level map analysis

Distributed System

complex system

real world

Vision

• A Network that optimizes itself:

– every device with a measurement module. – How to concert the measurements? – How to aggregate them? – How to analyze them is a hierarchical fashion?

14

DIMES Future

• DIMES as a leading research tool (6M measurements/day)

– Data is available to all – Easy to run distributed experiments

• Fast deploy cycle

– Easy to add new capabilities

• Packet Trains (joint effort with ETOMIC)
• Plug-ins to improve applications

– P2P communication – Web download (FireFox plug-in is available)

Current Status

• Over 3400 users, over 6000 agents

– 80 countries – All continents – Over 570 ASes – More than 1000 are active daily

• Over 6,000,000 measurements a day

15

Why PlanetLab?

DIMES has more agents and better spread than PlanetLab… … But, we do not want our agents to listen on ports

– Expose them to attacks – Problems with firewalls

ETOMIC serves as a high precision packet sink

– Only a few ETOMIC nodes are deployed – They are spread only in Europe

• PlanetLab will provide sinks world wide

– Challenge: Timing precision not as good as ETOMIC

Users by Country

(null) Albania Argentina Australia Austria Belarus Belgium Bermuda Brazil Bulgaria Canada Chile China Colombia Costa Rica Croatia Cyprus Czech Republic Denmark Egypt Estonia Finland France Germany Greece Guinea

Sp Aus Ger June 2005

16

How representative in PlanetLab?

Experiment under work: Compare groups of university agents with private/business agents:

• Delay distribution
• Delay stability
• Routing stability

Who

• PI: Yuval Shavitt
• Ph.D. students: Eran Shir, Tomer Tankel, Amir

Shay

• Master’s student: Galit Hadad, Dima Feldman, ..
• Programmers: Anat Halpern, Ohad Serfati
• Undergrads: Roni Ilani, Shay
• Collaborators: HUJI, ColBud

17

http://www.netdimes.org