SVM Learning of IP Address Structure for Latency Prediction Rob - - PowerPoint PPT Presentation

▶

May 04, 2023 409 likes •809 views

SVM Learning of IP Address Structure for Latency Prediction Rob Beverly, Karen Sollins and Arthur Berger {rbeverly,sollins,awberger}@csail.mit.edu SIGCOMM MineNet06 1 SVM Learning of IP Address Structure for Latency Prediction The case

SLIDE 1

SIGCOMM MineNet06 1

SVM Learning of IP Address Structure for Latency Prediction

Rob Beverly, Karen Sollins and Arthur Berger

{rbeverly,sollins,awberger}@csail.mit.edu

SLIDE 2

SIGCOMM MineNet06 2

SVM Learning of IP Address Structure for Latency Prediction

The case for Latency Prediction
The case for Machine Learning
Data and Methodology
Results
Going Forward

SLIDE 3

SIGCOMM MineNet06 3

The Case for Latency Prediction

SLIDE 4

SIGCOMM MineNet06 4

Latency Prediction (again?)

Significant prior work:

– King [Gummandi 2002] – Vivaldi [Dabek 2004] – Meridian [Wong 2005] – Others… IDMaps, GNP, etc…

Prior Methods:

– Active Queries – Synthetic Coordinate Systems – Landmarks

Our work seeks to provide an agent-centric

(single-node) alternative

SLIDE 5

SIGCOMM MineNet06 5

Why Predict Latency?

1. Service Selection: balance load, optimize

performance, P2P replication

2. User-directed Routing: e.g. IPv6 with per-

provider logical interfaces

3. Resource Scheduling: Grid computing,

etc.

4. Network Inference: Measure additional

topological network properties

SLIDE 6

SIGCOMM MineNet06 6

An Agent-Centric Approach

Hypothesis: Two hosts within same sub

network likely have consistent congestion and latency

Registry allocation policies give network

structure – but fragmented and discontinuous

Formulate as a supervised learning problem
Given latencies to a set of (random)

destinations as training:

– predict_latency(unseen IP) – error = |predict(IP) – actual(IP)|

SLIDE 7

SIGCOMM MineNet06 7

The Case for Machine Learning

SLIDE 8

SIGCOMM MineNet06 8

Why Machine Learning?

Internet-scale Networks:

– Complex (high-dimensional) – Dynamic

Can accommodate and recover from

infrequent errors in probabilistic world

Traffic provides large and continuous

training base

SLIDE 9

SIGCOMM MineNet06 9

Candidate Tool: Support Vector Machine

Supervised learning (but amenable to online

learning)

Separate training set into two classes in most

general way

Main insight: find hyper-plane separator that

maximizes the minimum margin between convex hulls of classes

Second insight: if data is not linearly separable,

take to higher dimension

Result: generalizes well, fast, accommodate

unknown data structure

SLIDE 10

SIGCOMM MineNet06 10

SVMs – Maximize Minimum Margin

=positive examples =negative examples

Most Simple Case: 2 Classes in 2 Dimensions Linearly Separable

=support vector

SLIDE 11

SIGCOMM MineNet06 11

SVMs – Redefining the Margin

=positive examples =negative examples

The single new positive example redefines margin

SLIDE 12

SIGCOMM MineNet06 12

Non-SV Points don’t affect solution

=positive examples =negative examples

SLIDE 13

SIGCOMM MineNet06 13

IP Latency Non-Linearity

=10ms examples =200ms examples

IP Bit 0 IP Bit 1

SLIDE 14

SIGCOMM MineNet06 14

Higher Dimensions for Non-Linearity

2 Classes in 2 Dimensions NOT Linearly Separable

?? ?? ?? =10ms examples =200ms examples

SLIDE 15

SIGCOMM MineNet06 15

Kernel Function Φ

2 Classes in 3 Dimensions Linearly Separable

=10ms examples =200ms examples

SLIDE 16

SIGCOMM MineNet06 16

Support Vector Regression

Same idea as classification
ε-insensitive loss function

First Octet Latency Φ Latency ε

SLIDE 17

SIGCOMM MineNet06 17

Data and Methodology

SLIDE 18

SIGCOMM MineNet06 18

Data Set

30,000 random hosts responding to ping
Average latency to each over 5 pings
Non-trivial distribution for learning

SLIDE 19

SIGCOMM MineNet06 19

Methodology

IP Latency # IP Latency # Permute Order Train Test

Average 5 experiments:

– Randomly permute data set – Split data set into training / test points

Data Set

SLIDE 20

SIGCOMM MineNet06 20

Methodology

Average 5 experiments:

– Training data defines SVM – Performance on (unseen) test points – Each bit of IP an input feature

IP Latency # IP Latency # Permute Order Train Test

Build SVM Measure Performance

SLIDE 21

SIGCOMM MineNet06 21

Results

SLIDE 22

SIGCOMM MineNet06 22

Results

Spoiler: So, does it work?
Yes, within 30% for more than 75% of

predictions

Performance varies with selection of

parameters (multi-optimization problem)

– Training Size – Input Dimension – Kernel

SLIDE 23

SIGCOMM MineNet06 23

Training Size

SLIDE 24

SIGCOMM MineNet06 24

Question: Are MSB Better Predictors

Determine error versus number most

significant bits of test input IPs

SLIDE 25

SIGCOMM MineNet06 25

Question: Are MSB Better Predictors

Determine error versus number most

significant bits of test input IPs

Powerful result: majority of discriminatory power in first 8 bits

SLIDE 26

SIGCOMM MineNet06 26

Feature Selection

Use feature selection to determine which

individual bits of address contribute to discriminatory power of prediction

θi ← ← ← ← argminj V(f(θ θ θ θ,xj),y) ∀ ∀ ∀ ∀ xj !∈ ∈ ∈ ∈ θ1,…,θi-1

SLIDE 27

SIGCOMM MineNet06 27

Feature Selection

Note x-axis: cumulative bit “features”

SLIDE 28

SIGCOMM MineNet06 28

Feature Selection

Matches Intuition and Registry Allocations

SLIDE 29

SIGCOMM MineNet06 29

Performance

Given empirically optimal training size and

input features

How well can agents predict latency to

unknown destinations?

SLIDE 30

SIGCOMM MineNet06 30

Prediction Performance

Ideal

SLIDE 31

SIGCOMM MineNet06 31

Prediction Performance

Within 30% for >75%

f Predictions

SLIDE 32

SIGCOMM MineNet06 32

Prediction Performance

Within 30% for >75%

f Predictions

SLIDE 33

SIGCOMM MineNet06 33

Going Forward

SLIDE 34

SIGCOMM MineNet06 34

Future Research

How agents select training data (random,

BGP prefix, registry allocation, from TCP flows, etc)

How performance decays over time and

how often to retrain

Online, continuous learning

SLIDE 35

SIGCOMM MineNet06 35

Summary - Questions?

Major Results:

– An agent-centric approach to latency prediction – Validation of SVMs and Kernel Functions as a means to learn on the basis of Internet Addresses – Feature Selection analysis of IP address informational content in predicting latency – Latency estimation accuracy within 30% of true value for > 75% of data points

SLIDE 36

SIGCOMM MineNet06 36

Prediction Accuracy

Only ~25% of predictions accurate to within 5% of actual value

SLIDE 37

SIGCOMM MineNet06 37

Prediction Accuracy

About ~45% of predictions accurate to within 10% of true value But >85% of predictions accurate to within 50% of true value

SLIDE 38

SIGCOMM MineNet06 38

Prediction Accuracy

Bottom line: suitable for coarse-grained latency prediction