network traffic classification
play

Network Traffic Classification: From Theory To Practice Valentn - PowerPoint PPT Presentation

Mar 01, 2024 •642 likes •1.81k views

Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Network Traffic Classification: From Theory To Practice Valentn Carela-Espaol Advisor: Pere Barlet-Ros Co-Advisor: Josep Sol

  1. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Open Problem High research activity to find novel classification solutions during the last years However, their introduction in operational networks is limited 7 / 47

  2. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Open Problem High research activity to find novel classification solutions during the last years However, their introduction in operational networks is limited What is slowing down their introduction? Existing techniques do not completely meet real-world requirements from operational networks 7 / 47

  3. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contributions Fill the gap between operational network requirements and existing traffic classification solutions. The Deployment Problem The Maintenance Problem The Validation Problem 8 / 47

  4. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contribution 1 Existing solutions have non scalable deployments for operational networks DPI techniques need expensive dedicated hardware to access to the payload of each packet ML techniques need dedicated hardware to compute the features of each flow 9 / 47

  5. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contribution 1 Existing solutions have non scalable deployments for operational networks DPI techniques need expensive dedicated hardware to access to the payload of each packet ML techniques need dedicated hardware to compute the features of each flow How to make the deployment of existing techniques easier? Reducing the requirements necessary for the classification Allowing packet sampling in the classification 9 / 47

  6. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contribution 2 Existing solutions have not feasible maintenances for operational networks DPI techniques have to update periodically the set of signatures ML techniques have to retrain periodically the classification models 10 / 47

  7. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contribution 2 Existing solutions have not feasible maintenances for operational networks DPI techniques have to update periodically the set of signatures ML techniques have to retrain periodically the classification models How to make the maintenance of existing techniques easier? Reducing the cost of the periodical updates Automatic Computationally viable Without human intervention 10 / 47

  8. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contribution 3 Validation and comparison of existing techniques are very difficult Different techniques Different datasets Different ground-truth generators 11 / 47

  9. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contribution 3 Validation and comparison of existing techniques are very difficult Different techniques Different datasets Different ground-truth generators How to make the validation of existing techniques easier? Validation of well-known ground-truth generators Publication of labeled datasets to the research community 11 / 47

  10. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Outline Introduction 1 Open Problem and Contributions 2 The Deployment Problem 3 The Maintenance Problem 4 The Validation Problem 5 Conclusions 6 11 / 47

  11. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions The Deployment Problem Existing solutions usually rely on packet level data 12 / 47

  12. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions The Deployment Problem Existing solutions usually rely on packet level data How to facilitate the deployment of existing techniques? Using Netflow (or SFlow, IPFIX) as input for the classification Limited amount of data available for the classification Being resilient to packet sampling What is the impact of packet sampling on existing techniques? 12 / 47

  13. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Methodology Using NetFlow v5 features (i.e., source and destination port, protocol, ToS, TCP flags, duration , # packets, # bytes, avg. packet size, avg. inter-arrival time) Technique based on the C4.5 decision tree Training Phase Validation Phase Online Classification NetFlow Packet Labelling Model building feature Traces process (WEKA) extraction Classification NetFlow v5 Classification model parser (C4.5) output NetFlow enabled router 13 / 47

  14. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Results with unsampled NetFlow Using the UPC dataset that we published Seven traces from UPC BarcelonaTech network Collected at different days and hours Labeled by a strict version of L7-filter (i.e., less false positives) Overall accuracy Port-based 8 Name C4.5 Flows Packets Bytes Flows UPC-I 89.17% 66.37% 56.53% 11.05% UPC-II 93.67% 82.04% 77.97% 11.68% UPC-III 90.77% 67.78% 61.80% 9.18% UPC-IV 91.12% 72.58% 63.69% 9.84% UPC-V 89.72% 70.21% 61.21% 6.49% UPC-VI 88.89% 68.48% 60.08% 16.98% UPC-VII 90.75% 61.37% 40.93% 3.55% Overall accuracy with unsampled NetFlow data 8 Internet Assigned Numbers Authority (IANA).http://www.iana.org/assignments/port-numbers, as of August 12, 2008. 14 / 47

  15. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Results with sampled NetFlow Impact of packet sampling on the classification Flow overall accuracy with sampled NetFlow data 15 / 47

  16. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of packet sampling Elements affected by packet sampling: Estimation of traffic features 1 o + 0.8 + + + o + + + Overall accuracy 0.6 o o o o o 0.4 0.2 o Estimated Features + Real Features 0.0 1 0.5 0.1 0.05 0.01 0.005 0.001 Sampling rate(p) Overall accuracy when removing the error introduced by the inversion of the features (UPC-I trace, using UPC-II for training) 16 / 47

  17. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of packet sampling Elements affected by packet sampling: Estimation of traffic features Flow size distribution (i.e., less mice flows) 0.35 p = 1 p = 0.1 p = 0.01 0.3 p = 0.001 0.25 0.2 Probability 0.15 0.1 0.05 0 0 10 20 30 40 50 60 70 80 90 100 Flow Length (packets) Flow length distribution of the detected flows when using several sampling probabilities. 17 / 47

  18. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of packet sampling Elements affected by packet sampling: Estimation of traffic features Flow size distribution (i.e., less mice flows) Flow splitting (i.e., split of elephant flows) 18000 Empirical Analytical 16000 14000 12000 Number of splits 10000 8000 6000 4000 2000 0 0.5 0.1 0.05 0.01 0.005 0.001 Sampling Rate (p) Amount of split flows as a function of the sampling probability p (UPC-II trace) 18 / 47

  19. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of packet sampling Elements affected by packet sampling: Estimation of traffic features Flow size distribution (i.e., less mice flows) Flow splitting (i.e., split of elephant flows) How to improve the accuracy under packet sampling? Applying the same packet sampling rate in the training phase 19 / 47

  20. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Improvement under packet sampling Improvement using the same packet sampling rate in the training phase Improvement of overall accuracy under packet sampling 20 / 47

  21. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Summary of the Deployment Problem We studied the impact of packet sampling on the classification Errors in: estimation of traffic features, flow size distribution, flow splitting We proposed a simple but effective technique to improve the classification accuracy under packet sampling We obtained a traffic classification solution easy to deploy Based on C4.5. decision tree Just using the limited information provided by NetFlow data Resilient to packet sampling 21 / 47

  22. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Outline Introduction 1 Open Problem and Contributions 2 The Deployment Problem 3 The Maintenance Problem 4 The Validation Problem 5 Conclusions 6 21 / 47

  23. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions The Maintenance Problem Using sampled NetFlow data as input we facilitate the deployment but, existing techniques still need periodic updates that hinder their maintenance How to perform the updates? How often is necessary to update the classifiers? 22 / 47

  24. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions The Maintenance Problem Using sampled NetFlow data as input we facilitate the deployment but, existing techniques still need periodic updates that hinder their maintenance How to perform the updates? How often is necessary to update the classifiers? How to facilitate the maintenance of existing techniques? 1st Approach: Autonomic Traffic Classification System 2nd Approach: Streaming-based Traffic Classification System 22 / 47

  25. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions 1st Approch: Autonomic Traffic Classification System Combining three different classification techniques (i.e., C5.0. decision tree, Service-based technique, IP-based technique) Relying in three DPI-based techniques for the ground-truth generation (i.e., PACE, OpenDPI, L7-filter) Using NetFlow v5 as input for the classification Classification path APPLICATION MONITORING TOOL IDENTIFIER NETFLOW v5 DATA A. I. LIB STORE UPDATE LABELED FLOWS FLOW NEW FLOW FEATURES TRAINING DATA STATISTICS TRAINED EXTRACTION Training PACKETS MODELS path RETRAINING A. I. LIB TRAINER MANAGER BUILDER ACTIVATE FLOW SAMPLING TRAINING (FULL PACKET) PACKETS FLOW LABELED LABELLING FLOWS CLASSIFY FLOWS A. I. LIB AUTONOMIC APPLICATION UPDATE IDENTIFIER RETRAINING SYSTEM 23 / 47

  26. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of the Autonomic Retraining System Overall accuracy with the CESCA dataset (i.e., 14 days long trace collected at the Catalan RREN) 100% 98% Accuracy 96% 94% Avg. accuracy = 96.76 % -- 5 retrainings -- 94% threshold Avg. accuracy = 97.5 % -- 15 retrainings -- 96% threshold Avg. accuracy = 98.26 % -- 108 retrainings -- 98% threshold Fri, 04 Feb 2011 Tue, 08 Feb 2011 Fri, 11 Feb 2011 Mon, 14 Feb 2011 Thu, 17 Feb 2011 Time Overall accuracy without sampling 24 / 47

  27. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of the Autonomic Retraining System (II) 100% 98% Accuracy 96% 94% Avg. accuracy = 96.65 % -- 5 retrainings -- 94% threshold Avg. accuracy = 97.34 % -- 17 retrainings -- 96% threshold Avg. accuracy = 98.22 % -- 116 retrainings -- 98% threshold Fri, 04 Feb 2011 Tue, 08 Feb 2011 Fri, 11 Feb 2011 Mon, 14 Feb 2011 Thu, 17 Feb 2011 Time Overall accuracy with 1/1000 sampling rate 25 / 47

  28. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of the Autonomic Retraining System (III) Comparative of the Autonomic Retraining System with static evaluations from existing solutions 100% 98% 96% 94% Accuracy 92% 90% 88% Avg. accuracy = 92.73 % -- trained with UPC-II Avg. accuracy = 94.3 % -- trained with first 3M CESCA flows Avg. accuracy = 98.24 % -- 108 retrainings -- 98% threshold, naive training policy with 500K Fri, 04 Feb 2011 Tue, 08 Feb 2011 Fri, 11 Feb 2011 Mon, 14 Feb 2011 Thu, 17 Feb 2011 Time Overall accuracy without sampling 26 / 47

  29. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Impact of the Autonomic Retraining System (IV) 100% 98% 94% 90% 86% Accuracy 82% 78% 74% 70% 66% Avg. accuracy = 68.38 % -- trained with UPC-II Avg. accuracy = 94.25 % -- trained with first 3M CESCA flows Avg. accuracy = 98.22 % -- 116 retrainings -- 98% threshold, naive training policy with 500K Fri, 04 Feb 2011 Tue, 08 Feb 2011 Fri, 11 Feb 2011 Mon, 14 Feb 2011 Thu, 17 Feb 2011 Time Overall accuracy with 1/1000 sampling rate in the CESCA dataset 27 / 47

  30. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Maintenance Problem (Improvement) 2nd Approach: Streaming-based Traffic Classification System Based on the stream-based ML technique Hoeffding Adaptive Tree Stream-based ML features: It processes a flow at a time and inspects it only once (in a single pass) It uses a limited amount of memory It works in a limited and small amount of time It is ready to predict at any time Uses Adaptive Sliding Window (ADWIN) to automatically adapt to the traffic changes Using NetFlow v5 as input for the classification 28 / 47

  31. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions HAT Evaluation Evaluation with the MAWI dataset (13 years of traffic from a transatlantic link in Japan) Interleaved Chunks Evaluation 100 80 % Accuracy 60 40 20 HAT J48 0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 # Flows Overall accuracy of HAT vs J48 (open-source C4.5) 29 / 47

  32. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions HAT Evaluation (II) Interleaved Chunks by Chunk Size 100 80 % Accuracy 60 40 20 HAT J48 0 0 1 2 3 4 5 6 10 10 10 10 10 10 10 Chunk Size Chunk size evaluation 30 / 47

  33. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions HAT Evaluation (III) Interleaved Chunks by Chunk Size Accumulated Cost 1.2 HAT_1000000 HAT_1000 1.0 HAT_1 J48_1000000 Cost (Gb per hour) J48_1000 0.8 J48_1 0.6 0.4 0.2 0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 1e7 # Flows Chunk size cost evaluation 31 / 47

  34. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Summary of the Maintenance Problem Existing classifiers need periodic retrainings Temporal obsolescence: evolution of the applications in the traffic Spatial obsolescence: different traffic mix We propose two solutions for traffic classification in operational networks: The Autonomic Traffic Classification System Easy to deploy: using sampled NetFlow Easy to maintain: thanks to the Autonomic Retraining System Accurate: combining three techniques (i.e., C5.0., Service-based technique and IP-based technique) HAT-base for traffic classification in operational networks: Easy to deploy: using sampled NetFlow Easy to maintain: automatically adapts to traffic changes with ADWIN Less cost than batch techniques and similar accuracy Limited use of memory No data is stored 32 / 47

  35. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Outline Introduction 1 Open Problem and Contributions 2 The Deployment Problem 3 The Maintenance Problem 4 The Validation Problem 5 Conclusions 6 32 / 47

  36. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions The Validation Problem Once addressed the deployment and maintenance problem, what technique do we select among the existing solutions? There are three main reasons that complicate the comparison and validation of the proposed solutions: Different techniques The solutions rely on different techniques (e.g., ML-based techniques, DPI-based techniques, Host-based techniques) Different datasets Solutions are usually evaluated with private datasets that cannot be shared because privacy issues. Different ground-truth generators The solutions use different techniques to label the datasets (e.g., DPI-based techniques) 33 / 47

  37. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Contributions Two main contributions in order to address the validation and comparison problem of network traffic classification solutions. Validation of different DPI-based techniques usually used as 1 ground-truth generators Publication of a reliable labeled dataset with full payload 2 34 / 47

  38. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Methodology How to publish a reliable labeled dataset with full payload? Reliable labeling of the dataset In order to properly label the traffic we rely on VBS 9 : Demon that extracts the label from the application that opens the socket for the communication Avoid privacy issues The content of the dataset is artificially created allowing its publication with full payload 6 Volunteer-Based System for Research on the Internet (2012) URL: http://vbsi.sourceforge.net/ 35 / 47

  39. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Methodology Three virtual machines with different OS running VBS Manually create the artificial traffic trying to be as representative as possible: Creating fake accounts (e.g., Gmail, Facebook, Twitter) Representing different human behaviors (e.g., posting, chatting, watching videos, playing games) 36 / 47

  40. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions The Dataset The dataset contains a total of 535438 flows and 32,61 GB of reliable labeled data. Application # Flows # Megabytes Edonkey 176581 2823.88 BitTorrent 62845 2621.37 FTP 876 3089.06 DNS 6600 1.74 NTP 27786 4.03 RDP 132907 13218.47 NETBIOS 9445 5.17 SSH 26219 91.80 Browser HTTP 46669 5757.32 Browser RTMP 427 5907.15 Unclassified 771667 3026.57 37 / 47

  41. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions DPI Evaluation Validation of 6 well-known DPI-based techniques for ground-truth generation Using the reliable dataset we previously built Name Version Applications PACE 1.41 (June 2012) 1000 OpenDPI 1.3.0 (June 2011) 100 nDPI rev. 6391 (March 2013) 170 L7-filter 2009.05.28 (May 2009) 110 Libprotoident 2.0.6 (Nov 2012) 250 NBAR 15.2(4)M2 (Nov 2012) 85 38 / 47

  42. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Results Application Classifier % correct % wrong % uncl. Application Classifier % correct % wrong % uncl. PACE 94.80 0.02 5.18 PACE 95.57 0.00 4.43 OpenDPI 0.45 0.00 99.55 OpenDPI 95.59 0.00 4.41 Edonkey L7-filter 34.21 13.70 52.09 SSH L7-filter 95.71 0.00 4.29 nDPI 0.45 6.72 92.83 nDPI 95.59 0.00 4.41 Libprotoident 98.39 0.00 1.60 Libprotoident 95.71 0.00 4.30 NBAR 0.38 10.81 88.81 NBAR 99.24 0.05 0.70 PACE 81.44 0.01 18.54 PACE 99.04 0.02 0.94 OpenDPI 27.23 0.00 72.77 OpenDPI 99.07 0.02 0.91 BitTorrent L7-filter 42.17 8.78 49.05 RDP L7-filter 0.00 91.21 8.79 nDPI 56.00 0.43 43.58 nDPI 99.05 0.08 0.87 Libprotoident 77.24 0.06 22.71 Libprotoident 98.83 0.16 1.01 NBAR 27.44 1.49 71.07 NBAR 0.00 0.66 99.34 PACE 95.92 0.00 4.08 PACE 66.66 0.08 33.26 OpenDPI 96.15 0.00 3.85 OpenDPI 24.63 0.00 75.37 FTP L7-filter 6.11 93.31 0.57 NETBIOS L7-filter 0.00 8.45 91.55 nDPI 95.69 0.45 3.85 nDPI 100.00 0.00 0.00 Libprotoident 95.58 0.00 4.42 Libprotoident 0.00 5.03 94.97 NBAR 40.59 0.00 59.41 NBAR 100.00 0.00 0.00 PACE 99.97 0.00 0.03 PACE 80.56 0.00 19.44 OpenDPI 99.97 0.00 0.03 OpenDPI 82.44 0.00 17.56 DNS L7-filter 98.95 0.13 0.92 RTMP L7-filter 0.00 24.12 75.88 nDPI 99.88 0.09 0.03 nDPI 78.92 8.90 12.18 Libprotoident 99.97 0.00 0.04 Libprotoident 77.28 0.47 22.25 NBAR 99.97 0.02 0.02 NBAR 0.23 0.23 99.53 PACE 100.00 0.00 0.00 PACE 96.16 1.85 1.99 OpenDPI 100.00 0.00 0.00 OpenDPI 98.01 0.00 1.99 NTP L7-filter 99.83 0.15 0.02 HTTP L7-filter 4.31 95.67 0.02 nDPI 100.00 0.00 0.00 nDPI 99.18 0.76 0.06 Libprotoident 100.00 0.00 0.00 Libprotoident 98.66 0.00 1.34 NBAR 0.40 0.00 99.60 NBAR 99.58 0.00 0.42 39 / 47

  43. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Results Application Classifier % correct % wrong % uncl. Application Classifier % correct % wrong % uncl. PACE 94.80 0.02 5.18 PACE 95.57 0.00 4.43 OpenDPI 0.45 0.00 99.55 OpenDPI 95.59 0.00 4.41 Edonkey L7-filter 34.21 13.70 52.09 SSH L7-filter 95.71 0.00 4.29 nDPI 0.45 6.72 92.83 nDPI 95.59 0.00 4.41 Libprotoident 98.39 0.00 1.60 Libprotoident 95.71 0.00 4.30 NBAR 0.38 10.81 88.81 NBAR 99.24 0.05 0.70 PACE 81.44 0.01 18.54 PACE 99.04 0.02 0.94 OpenDPI 27.23 0.00 72.77 OpenDPI 99.07 0.02 0.91 BitTorrent L7-filter 42.17 8.78 49.05 RDP L7-filter 0.00 91.21 8.79 nDPI 56.00 0.43 43.58 nDPI 99.05 0.08 0.87 Libprotoident 77.24 0.06 22.71 Libprotoident 98.83 0.16 1.01 NBAR 27.44 1.49 71.07 NBAR 0.00 0.66 99.34 PACE 95.92 0.00 4.08 PACE 66.66 0.08 33.26 OpenDPI 96.15 0.00 3.85 OpenDPI 24.63 0.00 75.37 FTP L7-filter 6.11 93.31 0.57 NETBIOS L7-filter 0.00 8.45 91.55 nDPI 95.69 0.45 3.85 nDPI 100.00 0.00 0.00 Libprotoident 95.58 0.00 4.42 Libprotoident 0.00 5.03 94.97 NBAR 40.59 0.00 59.41 NBAR 100.00 0.00 0.00 PACE 99.97 0.00 0.03 PACE 80.56 0.00 19.44 OpenDPI 99.97 0.00 0.03 OpenDPI 82.44 0.00 17.56 DNS L7-filter 98.95 0.13 0.92 RTMP L7-filter 0.00 24.12 75.88 nDPI 99.88 0.09 0.03 nDPI 78.92 8.90 12.18 Libprotoident 99.97 0.00 0.04 Libprotoident 77.28 0.47 22.25 NBAR 99.97 0.02 0.02 NBAR 0.23 0.23 99.53 PACE 100.00 0.00 0.00 PACE 96.16 1.85 1.99 OpenDPI 100.00 0.00 0.00 OpenDPI 98.01 0.00 1.99 NTP L7-filter 99.83 0.15 0.02 HTTP L7-filter 4.31 95.67 0.02 nDPI 100.00 0.00 0.00 nDPI 99.18 0.76 0.06 Libprotoident 100.00 0.00 0.00 Libprotoident 98.66 0.00 1.34 NBAR 0.40 0.00 99.60 NBAR 99.58 0.00 0.42 39 / 47

  44. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Results Application Classifier % correct % wrong % uncl. Application Classifier % correct % wrong % uncl. PACE 94.80 0.02 5.18 PACE 95.57 0.00 4.43 OpenDPI 0.45 0.00 99.55 OpenDPI 95.59 0.00 4.41 Edonkey L7-filter 34.21 13.70 52.09 SSH L7-filter 95.71 0.00 4.29 nDPI 0.45 6.72 92.83 nDPI 95.59 0.00 4.41 Libprotoident 98.39 0.00 1.60 Libprotoident 95.71 0.00 4.30 NBAR 0.38 10.81 88.81 NBAR 99.24 0.05 0.70 PACE 81.44 0.01 18.54 PACE 99.04 0.02 0.94 OpenDPI 27.23 0.00 72.77 OpenDPI 99.07 0.02 0.91 BitTorrent L7-filter 42.17 8.78 49.05 RDP L7-filter 0.00 91.21 8.79 nDPI 56.00 0.43 43.58 nDPI 99.05 0.08 0.87 Libprotoident 77.24 0.06 22.71 Libprotoident 98.83 0.16 1.01 NBAR 27.44 1.49 71.07 NBAR 0.00 0.66 99.34 PACE 95.92 0.00 4.08 PACE 66.66 0.08 33.26 OpenDPI 96.15 0.00 3.85 OpenDPI 24.63 0.00 75.37 FTP L7-filter 6.11 93.31 0.57 NETBIOS L7-filter 0.00 8.45 91.55 nDPI 95.69 0.45 3.85 nDPI 100.00 0.00 0.00 Libprotoident 95.58 0.00 4.42 Libprotoident 0.00 5.03 94.97 NBAR 40.59 0.00 59.41 NBAR 100.00 0.00 0.00 PACE 99.97 0.00 0.03 PACE 80.56 0.00 19.44 OpenDPI 99.97 0.00 0.03 OpenDPI 82.44 0.00 17.56 DNS L7-filter 98.95 0.13 0.92 RTMP L7-filter 0.00 24.12 75.88 nDPI 99.88 0.09 0.03 nDPI 78.92 8.90 12.18 Libprotoident 99.97 0.00 0.04 Libprotoident 77.28 0.47 22.25 NBAR 99.97 0.02 0.02 NBAR 0.23 0.23 99.53 PACE 100.00 0.00 0.00 PACE 96.16 1.85 1.99 OpenDPI 100.00 0.00 0.00 OpenDPI 98.01 0.00 1.99 NTP L7-filter 99.83 0.15 0.02 HTTP L7-filter 4.31 95.67 0.02 nDPI 100.00 0.00 0.00 nDPI 99.18 0.76 0.06 Libprotoident 100.00 0.00 0.00 Libprotoident 98.66 0.00 1.34 NBAR 0.40 0.00 99.60 NBAR 99.58 0.00 0.42 39 / 47

  45. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Results Application Classifier % correct % wrong % uncl. Application Classifier % correct % wrong % uncl. PACE 94.80 0.02 5.18 PACE 95.57 0.00 4.43 OpenDPI 0.45 0.00 99.55 OpenDPI 95.59 0.00 4.41 Edonkey L7-filter 34.21 13.70 52.09 SSH L7-filter 95.71 0.00 4.29 nDPI 0.45 6.72 92.83 nDPI 95.59 0.00 4.41 Libprotoident 98.39 0.00 1.60 Libprotoident 95.71 0.00 4.30 NBAR 0.38 10.81 88.81 NBAR 99.24 0.05 0.70 PACE 81.44 0.01 18.54 PACE 99.04 0.02 0.94 OpenDPI 27.23 0.00 72.77 OpenDPI 99.07 0.02 0.91 BitTorrent L7-filter 42.17 8.78 49.05 RDP L7-filter 0.00 91.21 8.79 nDPI 56.00 0.43 43.58 nDPI 99.05 0.08 0.87 Libprotoident 77.24 0.06 22.71 Libprotoident 98.83 0.16 1.01 NBAR 27.44 1.49 71.07 NBAR 0.00 0.66 99.34 PACE 95.92 0.00 4.08 PACE 66.66 0.08 33.26 OpenDPI 96.15 0.00 3.85 OpenDPI 24.63 0.00 75.37 FTP L7-filter 6.11 93.31 0.57 NETBIOS L7-filter 0.00 8.45 91.55 nDPI 95.69 0.45 3.85 nDPI 100.00 0.00 0.00 Libprotoident 95.58 0.00 4.42 Libprotoident 0.00 5.03 94.97 NBAR 40.59 0.00 59.41 NBAR 100.00 0.00 0.00 PACE 99.97 0.00 0.03 PACE 80.56 0.00 19.44 OpenDPI 99.97 0.00 0.03 OpenDPI 82.44 0.00 17.56 DNS L7-filter 98.95 0.13 0.92 RTMP L7-filter 0.00 24.12 75.88 nDPI 99.88 0.09 0.03 nDPI 78.92 8.90 12.18 Libprotoident 99.97 0.00 0.04 Libprotoident 77.28 0.47 22.25 NBAR 99.97 0.02 0.02 NBAR 0.23 0.23 99.53 PACE 100.00 0.00 0.00 PACE 96.16 1.85 1.99 OpenDPI 100.00 0.00 0.00 OpenDPI 98.01 0.00 1.99 NTP L7-filter 99.83 0.15 0.02 HTTP L7-filter 4.31 95.67 0.02 nDPI 100.00 0.00 0.00 nDPI 99.18 0.76 0.06 Libprotoident 100.00 0.00 0.00 Libprotoident 98.66 0.00 1.34 NBAR 0.40 0.00 99.60 NBAR 99.58 0.00 0.42 39 / 47

  46. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Results (II) HTTP Sub-Classification with nDPI Application % correct % wrong % unclassified Google 97.28 2.72 0.00 Facebook 100.00 0.00 0.00 Youtube 98.65 0.45 0.90 Twitter 99.75 0.00 0.25 FLASH over HTTP Evaluation Classifier % correct % wrong % unclassified PACE 86.27 13.18 0.55 OpenDPI 86.34 13.15 0.51 L7-filter 0.07 99.67 0.26 nDPI 99.48 0.26 0.26 Libprotoident 0.00 98.07 1.93 NBAR 0.00 100.00 0.00 40 / 47

  47. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Summary of the Validation Problem Summary results: Classifier % Precision % Avg. Precision PACE 94.22 91.01 OpenDPI 52.67 72.35 L7-filter 30.26 38.13 nDPI 57.91 82.48 Libprotoident 93.86 84.16 NBAR 21.79 46.72 PACE is the most reliable tool for ground-truth generation. nDPI and Libprotoident are the most reliable open-source tools. nDPI is recommended for sub-classification evaluation Libprotoident for scenarios with truncated traffic (e.g., 96 bytes of payload) NBAR and L7-filter are not recommended in their current form. 41 / 47

  48. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Outline Introduction 1 Open Problem and Contributions 2 The Deployment Problem 3 The Maintenance Problem 4 The Validation Problem 5 Conclusions 6 41 / 47

  49. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Conclusions Addressed important practical challenges of existing techniques for network traffic classification in operational networks The Deployment Problem studied traffic classification using NetFlow data as input studied the impact of packet sampling on traffic classification The Maintenance Problem showed that classification models suffer from temporal and spatial obsolescence addressed this problem by: proposing a complete traffic classification solution with a novel automatic retraining system introduce the use of the stream-base ML Hoeffding Adaptive Tree for traffic classification The Validation Problem compared 6 well-known DPI-based tools for ground-truth generation. published a reliable labeled dataset with full payload 9 9 http://www.cba.upc.edu/monitoring/traffic-classification 42 / 47

  50. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Future Work The Deployment and Maintenance Problem Use of NBAR2 for the retraining The Validation Problem Study of new applications Study of new DPI-based tools (e.g., NBAR2) The Network Traffic Classification Problem Multilabel classification Distributed solutions: Hadoop and SAMOA 43 / 47

  51. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Related Publications Journals: V. Carela-Español , P . Barlet-Ros, A. Bifet and K. Fukuda. “ A streaming flow-based technique for traffic classification applied to 12+1 years of Internet traffic ”. Journal of Telecommunications Systems, 2014.(Under Review) T. Bujlow, V. Carela-Español and P . Barlet-Ros. “ Independent Comparison of Popular DPI Tools for Trac Classification ”. Computer Networks, 2014 (Under Review) V. Carela-Español , P . Barlet-Ros, O. Mula-Valls and J. Solé-Pareta. “ An automatic traffic classification system for network operation and management ”. Journal of Network and Systems Management, October 2013. V. Carela-Español , P . Barlet-Ros, A. Cabellos-Aparicio, and J. Solé-Pareta. “ Analysis of the impact of sampling on NetFlow traffic classification ”. Computer Networks 55 (2011), pp. 1083-1099. Conferences: V. Carela-Español , T. Bujlow, and P . Barlet-Ros. “ Is Our Ground-Truth for Traffic Classification Reliable? ”. In Proc. of the Passive and Active Measurements Conference (PAM’14), Los Angeles, CA, USA, March 2014. J. Molina, V. Carela-Español , R. Hoffmann, K. Degner and P . Barlet-Ros. “ Empirical analysis of traffic to establish a profiled flow termination timeout ”. In Proc. of Intl. Workshop on Traffic Analysis and Classification (TRAC), Cagliari, Italy, July 2013. V. Carela-Español , P . Barlet-Ros, M. Solé-Simó, A. Dainotti, W. de Donato and A. Pesacapé. “ K-dimensional trees for continuous traffic classification ”. In Proc. of Second International Workshop on Traffic Monitoring and Analysis. Zurich, Switzerland, April 2010. (COST Action IC0703) P . Barlet-Ros, V. Carela-Español , E. Codina and J. Solé-Pareta. “ Identification of Network Applications based on Machine Learning Techniques ”. In Proc. of TERENA Networking Conference. Brugge, Belgium, May 2008. Supervised Master Students Juan Molina Rodriguez : “ Empirical analysis of traffic to establish a profiled flow termination timeout ”, 2013 in collaboration with ipoque. Datasets UPC Dataset: NetFlow v5 dataset labeled by L7-filter PAM Dataset: full packet payload dataset labeled by VBS 44 / 47

  52. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions UPC Dataset Name Supervisor Institution Date Giantonio Chiarelli Domenico Vitali Universita degli Studi di Roma La Jan 2011 Sapienza (Rome, Italy) Sanping Li - University of Massachusetts Lowell Feb 2011 (Lowell, USA) Qian Yaguan Wu Chunming CS College of Zhejiang University Apr 2011 (Hangzhou, China) Yulios Zabala Lee Luan Ling State University of Campinas - Uni- Aug 2011 camp (So Paulo, Brazil) Massimiliano Natale Domenico Vitali Universita degli Studi di Roma La Jan 2012 Sapienza (Rome, Italy) Elie Bursztein - Stanford University (Stanford, USA) Feb 2012 Jesus Diaz Verdejo - Universidad de Granada (Granada, Feb 2013 Spain) Ning Gao Quin Lv University of Colorado Boulder Feb 2013 (Boulder, USA) Wesley Melo Stenio Fernandes GPRT - Networking and Telecom- Jul 2013 munications Research Group (Re- cife, Brazil) Adriel Cheng - Department of Defence (Edinburgh, Sep 2013 Australia) Corey Hart - Lockheed Martin (King of Prussia, Oct 2013 PA, USA) Rajesh NP - Cisco (Bangalore, India) Dec 2013 Raja Rajendran Andrew Ng Stanford University (Stanford, USA) Dec 2013 Indranil Adak Raja Rajendran Cisco (Bangalore, India) Dec 2013 45 / 47

  53. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions PAM Dataset Name Supervisor Institution Date Said Sedikki Ye-Qiong Song University of Lorraine (Villers-les- Mar 2014 Nancy, France) Oliver Gasser Georg Carle Technische Universitat Munchen Mar 2014 (Munchen, Germany) Viktor Minorov Pavel Celada Masaryk University (Brno, Czech Mar 2014 Republic) Yiyang Shao Jun Li Tsinghua University (Beijing, Apr 2014 China) Yinsen Miao Farinaz Koushanfar Rice University (Houston, USA) Apr 2014 Le Quoc Do Christof Fetzer Technische Universitat Dresden May 2014 (Dresden, Germany) Zuleika Nascimento Djamel Salok Federal University of Pernambuco May 2014 (Recife, Brazil) Garrett Cullity Adriel Cheng University of Adelaide (Adelaide, May 2014 Australia) Zeynab Sabahi Ahmad Nickabadi University of Tehran (Tehran, Iran) Jun 2014 Joseph Kampeas Omer Gurewitz Ben Gurion University of the Jun 2014 Negev (Beer Sheva, Israel) Hossein Doroud Andres Marin Univesidad Carlos III (Madrid, Jul 2014 Spain) Alioune BA Cedric Baudoin Thales Alenia Space (Toulouse, Jul 2014 France) Jan-Erik Stange - University of Applied Science Jul 2014 Postdam (Postdam, Germany) 46 / 47

  54. Introduction Contributions The Deployment Problem The Maintenance Problem The Validation Problem Conclusions Network Traffic Classification: From Theory To Practice Valentín Carela-Español Advisor: Pere Barlet-Ros Co-Advisor: Josep Solé Pareta Departament d’Arquitectura de Computadors Universitat Politècnica de Catalunya (UPC BarcelonaTech) October 31, 2014 UNIVERSITAT POLITÈCNICA D E P A R T A M E N T DE CATALUNYA D ’ A R Q U I T E C T U R A D E C O M P U T A D O R S 47 / 47

  55. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides Motivations Taxonomy of the proposed traffic classification techniques Technique High High Computationally Packet Content Easy to Deploy Easy to Maintain Accuracy Completeness Lightweight ✗ ✗ � � � � Well-known Ports � � / ✗ ✗ ✗ ✗ ✗ Pattern Matching (DPI) � / ✗ � / ✗ ✗ � � � Host-Behavior � � � � ✗ ✗ Machine Learning (ML) � � � � � / ✗ ✗ Service and IPs 48 / 47

  56. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Traffic breakdown of the traces in the UPC dataset 49 / 47

  57. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Set of 10 NetFlow-based features Feature Description Value sport Source port of the flow 16 bits dport Destination port of the flow 16 bits protocol IP protocol value 8 bits ToS Type of Service from the first packet 8 bits flags Cumulative OR of TCP flags 6 bits duration Duration of the flow in nsec precision ts end − ts ini packets packets Total number of packets in the flow p bytes bytes Flow length in bytes p bytes pkt _ size Average packet size of the flow packets duration iat Average packet inter-arrival time packets / p 50 / 47

  58. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Elephant flow distribution in the UPC dataset Name Flows Elephant Flows % Flows % Bytes UPC-I 2 985 K 0.035818% 52.17% UPC-II 3 369 K 0.048619% 61.45% UPC-III 3 474 K 0.041587% 59.58% UPC-IV 3 020 K 0.048149% 59.79% UPC-V 7 146 K 0.014151% 66.08% UPC-VI 9 718 K 0.042271% 54.51% UPC-VII 5 510 K 0.014075% 72.44% 51 / 47

  59. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Precision (mean with 95% CI) by application group (per flow) of our traffic classification method (C4.5) with different sampling rates 52 / 47

  60. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Flags and ToS: Under random sampling, the probability p of sampling a packet is independent from the other packets. Let m be the number of packets of a particular flow with the flag f set (i.e., f = 1), where f ∈ { 0 , 1 } . The probability of incorrectly estimating the value of f under sampling is ( 1 − p ) m , independently of how the packets with the flag set are distributed over the flow. The expected value of the absolute error is: f ] = f − ( 1 − ( 1 − p ) m ) = f − 1 + ( 1 − p ) m E [ f − ˆ f ] = f − E [ˆ (1) f is biased, since the expectation of the error is ( 1 − p ) m when f = 1, and it is only 0 when f = 0. That is, with packet Eq. 1 shows that ˆ sampling ˆ f tends to underestimate f , especially when f = 1 and m or p are small. For example, if we have a flow with 100 packets with the flag ACK set ( m = 100) and p = 1 % , the expectation of the error in the flag ACK is ( 1 − 0 . 01 ) 100 ≈ 0 . 37. The flag SYN and the ToS are particular cases, where we are only interested in the first packet and, therefore, m ∈ { 0 , 1 } . 53 / 47

  61. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Number of packets. With sampling probability p , the number of sampled packets x from a flow of n packets follows a binomial distribution x ∼ B ( n , p ) . Thus, the expected value of the estimated feature ˆ n = x / p is: � x � 1 1 E [ˆ n ] = E = E [ x ] = np = n (2) p p p which shows that ˆ n is an unbiased estimator of n (i.e., the expected value of the error is 0). The variance of ˆ n is: � x � 1 1 1 Var [ˆ n ] = Var = p 2 Var [ x ] = p 2 np ( 1 − p ) = n ( 1 − p ) (3) p p Hence, the variance of the relative error can be expressed as: � � � ˆ � n ˆ n 1 1 1 − p n 2 Var [ˆ Var 1 − = Var = n ] = n ( 1 − p ) = (4) n 2 p n n np Eq. 4 indicates that, for a given p , the variance of the error decreases with n . That is, the variance of the error for elephant flows is smaller than for smaller flows. The variance also increases when p is small. For example, with p = 1 % , the variance of the error of a flow with 100 1 − 0 . 01 100 × 0 . 01 = 0 . 99, which is not negligible. packets is 54 / 47

  62. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Flow size. The original size b of a flow is defined as b = � n i = 1 bi , where n is the total number of packets of the flow and bi is the size of each individual packet. Under random sampling, we can estimate b from a subset of sampled packets by renormalizing their size: n � bi ˆ b = wi (5) p i = 1 where wi ∈ { 0 , 1 } are Bernoulli distributed random variables with probability p . We can show that ˆ b is an unbiased estimator of b , since E [ˆ b ] = b : � n � n � � n � bi 1 � 1 � E [ˆ b ] = E wi = E wi bi = E [ wi bi ] = p p p i = 1 i = 1 i = 1 n n n 1 � 1 � 1 � = bi E [ wi ] = bi p = p bi = b (6) p p p i = 1 i = 1 i = 1 The variance of ˆ b is obtained as follows: � n � � n � n � bi 1 � 1 � Var [ˆ b ] = Var wi = p 2 Var wi bi = Var [ wi bi ] = p 2 p i = 1 i = 1 i = 1 n n n 1 � 1 � 1 − p � b 2 b 2 bi 2 = i Var [ wi ] = i p ( 1 − p ) = (7) p 2 p 2 p i = 1 i = 1 i = 1 Thus, the variance of the relative error is: � � � ˆ � ˆ n b b 1 1 1 − p � bi 2 = b 2 Var [ˆ Var 1 − = Var = b ] = b 2 b b p i = 1 � n i = 1 bi 2 1 − p = (8) � � n � 2 p i = 1 bi � 2. This indicates that the variance of the error can be significant for small sampling which decreases with n , since � n i bi 2 ≤ ( � n i bi 55 / 47

  63. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Duration and interarrival time. The flow duration is defined as d = tn − t 1, where t 1 and tn are the timestamps of the first and last packets of the original flow. Under sampling, this duration is estimated as ˆ d = tb − ta , where ta and tb are the timestamps of the first and last sampled packets respectively. Thus, the expected value of ˆ d is: E [ˆ d ] = E [ tb − ta ] = E [ tb ] − E [ ta ] � � � � n a � � = E tn − iati − E t 1 + iati i = b i = 1 � � n � � a �� � � = ( tn − t 1 ) − E iati + E iati (9) i = b i = 1 where iati is the interarrival time between packets i and i − 1, and a is a random variable that denotes the number of missed packets until the first packet of the flow is sampled (i.e., the number of packets between t 1 and ta ). Therefore, the variable a follows a geometric distribution with probability p , whose expectation is 1 / p . By symmetry, we can consider the number of packets between b and n to follow the same geometric distribution. In this case, we can rewrite Eq. 9 as follows: E [ˆ d ] = ( tn − t 1 ) − (( E [ n − b ] E [ iat ]) + ( E [ a ] E [ iat ])) � iat � = ( tn − t 1 ) − 2 (10) p where iat is the average interarrival time of the non-sampled packets. Eq. 10 shows that the estimated duration is biased (i.e., E [ d − ˆ d ] > 0). In other words, ˆ d always underestimates d . The bias is ( 2 × iat / p ) , if we consider the average interarrival time to be equal between packets 1 . . . a and b . . . n . However, we cannot use the feature � iat to correct this bias, because this feature is obtained directly from ˆ d . In fact, Eq. 10 indicates that the feature � iat is also biased, since � iat = ˆ d / ˆ n . 56 / 47

  64. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Average of the relative error of the flow features as a function of p (UPC-II trace) Feature p = 0.5 p = 0.1 p = 0.05 p = 0.01 p = 0.005 p = 0.001 sport 0.00 0.00 0.00 0.00 0.00 0.00 dport 0.00 0.00 0.00 0.00 0.00 0.00 proto 0.00 0.00 0.00 0.00 0.00 0.00 ˆ f 0.05 0.16 0.18 0.22 0.23 0.24 ˆ d 0.22 0.60 0.66 0.77 0.79 0.81 ˆ n 0.66 3.66 6.90 29.69 55.17 234.61 ˆ b 0.76 3.86 7.05 29.71 55.09 234.24 � iat 0.29 0.65 0.71 0.78 0.80 0.82 57 / 47

  65. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Validation against the empirical distribution of the original flow length detected with p = 0 . 1 (UPC-II trace). 0.35 Analytical PMF Empirical PMF 0.3 0.25 0.2 Probability 0.15 0.1 0.05 0 0 10 20 30 40 50 60 70 80 90 100 Flow Length (packets) 58 / 47

  66. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Deployment Problem Backup Slides Precision (mean with 95% CI) by application group (per flow) of our traffic classification method with a sampled training set 59 / 47

  67. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Application groups and traffic mix in the UPC-II and CESCA datasets Group Applications Flows UPC-II CESCA Web HTTP 678 863 17 198 845 DD E.g., Megaupload, MediaFire 2 168 40 239 Multimedia E.g., Flash, Spotify, Sopcast 20 228 1 126 742 P2P E.g., Bittorrent, eDonkey 877 383 4 851 103 Mail E.g., IMAP , POP3 19 829 753 075 Bulk E.g., FTP , AFTP 1 798 27 265 VoIP E.g., Skype, Viber 411 083 3 385 206 DNS DNS 287 437 15 863 799 chat E.g., Jabber, MSN Messenger 12 304 196 731 Games E.g., Steam, WoW 2 880 14 437 Encryption E.g., SSL, OpenVPN 71 491 3 440 667 Others E.g., Citrix, VNC 55 829 2 437 664 60 / 47

  68. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides DPI labeling contribution in the CESCA dataset PACE PACE L7_FILTER 12.16% L7_FILTER 17.25% 3.46% 4.41% 29.34% 26.65% 52.58% 49.23% 2.24% 2.24% 0.04% 0.04% 0.18% 0.18% OPENDPI OPENDPI 61 / 47

  69. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Different training policies: Long-Term and Naive Different training sizes: 100K, 500k, 1M flows Training Metric Training Policy Size Long-Term Policy Naive Policy Avg. Accuracy 97.57% 98.00% 100K Min. Accuracy 95.95% 97.01% # Retrainings 688 525 Avg. Training Time 88 s 25 s Avg. Accuracy 98.12% 98.26% 500K Min. Accuracy 95.44% 95.70% # Retrainings 125 108 Avg. Training Time 232 s 131 s Avg. Accuracy 98.18% 98.26% 1M Min. Accuracy 94.78% 94.89% # Retrainings 61 67 Avg. Training Time 485 s 262 s 62 / 47

  70. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Comparative of the Autonomic Retraining System by institution 100% 100% 100% 98% 98% 98% 96% 96% 96% Accuracy Accuracy Accuracy 94% 94% 94% 92% 92% 92% Avg. accuracy = 98.06 % -- 35 retrainings -- 98% threshold Avg. accuracy = 98.05 % -- 13 retrainings -- 98% threshold Avg. accuracy = 98.26 % -- 9 retrainings -- 98% threshold Avg. accuracy = 98.41 % -- 108 retrainings -- 98% threshold Avg. accuracy = 97.91 % -- 109 retrainings -- 98% threshold Avg. accuracy = 98.17 % -- 108 retrainings -- 98% threshold Fri, 04 Feb 2011 Tue, 08 Feb 2011 Fri, 11 Feb 2011 Mon, 14 Feb 2011 Thu, 17 Feb 2011 Fri, 04 Feb 2011 Tue, 08 Feb 2011 Fri, 11 Feb 2011 Mon, 14 Feb 2011 Thu, 17 Feb 2011 Fri, 04 Feb 2011 Tue, 08 Feb 2011 Fri, 11 Feb 2011 Mon, 14 Feb 2011 Thu, 17 Feb 2011 Time Time Time 63 / 47

  71. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Impact of the Numeric Estimator parameter on the HAT technique Evaluate Interleaved Chunks HAT Evaluate Interleaved Chunks HAT 100 4.5 VFML 10 4.0 VFML 100 VFML 1000 80 3.5 VFML 10000 Cost (Gb per hour) BT 3.0 GREEN (10,100) % Accuracy 60 GAUSS (10,100) 2.5 2.0 40 VFML 10 VFML 100 1.5 VFML 1000 VFML 10000 1.0 20 BT GREEN (10,100) 0.5 GAUSS (10,100) 0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 # Flows 1e7 # Flows 1e7 64 / 47

  72. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Impact of the Grace Period parameter on the HAT technique Evaluate Interleaved Chunks HAT Evaluate Interleaved Chunks HAT 100 0.16 Grace 5000 Grace 2000 0.14 Grace 1000 80 Grace 200 0.12 Cost (Gb per hour) Grace 50 0.10 % Accuracy 60 0.08 40 0.06 Grace 5000 0.04 Grace 2000 20 Grace 1000 0.02 Grace 200 Grace 50 0 0.00 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 # Flows 1e7 # Flows 1e7 65 / 47

  73. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Impact of the Tie Threshold parameter on the HAT technique Evaluate Interleaved Chunks HAT Evaluate Interleaved Chunks HAT 100 0.040 tie 1 tie 0.5 0.035 tie 0.25 80 tie 0.1 0.030 Cost (Gb per hour) tie 0.05 tie 0.001 0.025 % Accuracy 60 0.020 40 0.015 tie 1 tie 0.5 0.010 tie 0.25 20 tie 0.1 0.005 tie 0.05 tie 0.001 0 0.000 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 # Flows 1e7 # Flows 1e7 66 / 47

  74. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Impact of the Split Criteria parameter on the HAT technique Evaluate Interleaved Chunks HAT Evaluate Interleaved Chunks HAT 100 0.040 InfoGain 0.001 InfoGain 0.01 0.035 InfoGain 0.1 80 InfoGain 0.25 0.030 Cost (Gb per hour) InfoGain 0.5 Gini 0.025 % Accuracy 60 0.020 40 0.015 InfoGain 0.001 InfoGain 0.01 0.010 InfoGain 0.1 20 InfoGain 0.25 0.005 InfoGain 0.5 Gini 0 0.000 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 # Flows 1e7 # Flows 1e7 67 / 47

  75. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Impact of the Leaf Prediction parameter on the HAT technique Evaluate Interleaved Chunks HAT Evaluate Interleaved Chunks HAT 100 0.030 Majority Class Naive Bayes 0.025 Naive Bayes Adaptive 80 Cost (Gb per hour) 0.020 % Accuracy 60 0.015 40 0.010 20 Majority Class 0.005 Naive Bayes Naive Bayes Adaptive 0 0.000 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 # Flows 1e7 # Flows 1e7 68 / 47

  76. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides HAT parametrization for network traffic classification Parameter Value Numeric Estimator VFML with 1 000 bins Grace Period 1 000 instances (i.e., flows) Tie Threshold 1 Split Criteria Information Gain with 0.001 as minimum fraction of weight Leaf Prediction Majority Class Stop Memory Management Activated Binary Splits Activated Remove Poor Attributes Activated 69 / 47

  77. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides HAT comparison with single training configuration Single Training Evaluation 100 80 % Accuracy 60 40 20 HAT J48 0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Time 70 / 47

  78. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides HAT cost comparison by chunk size Interleaved Chunks by Chunk Size Cost by Flow 1 10 Cost (bytes per second) 0 10 -1 10 HAT_1000000 HAT_1000 HAT_1 J48_1000000 J48_1000 -2 J48_1 10 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 1e7 # Flows 71 / 47

  79. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Interleaved Chunk comparison with [8] configuration Periodic Training Evaluation 100 80 % Accuracy 60 40 20 HAT J48 [8] 0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Time 72 / 47

  80. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides The Maintenance Problem Backup Slides Interleaved Chunk evaluation with CESCA dataset Interleaved Chunks Evaluation 100 80 % Accuracy 60 40 20 HAT J48 0 0 1 2 3 4 5 6 1e7 # Flows 73 / 47

  81. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides Other Improvement Backup Slides Characteristics of the ISP traces in the evaluation dataset Duration Flows Packets Bytes ISP_Core 38 160 s 295 729 K 7 074 618 K 2 591 636 M ISP_Mob 2 700 s 6 093 K 233 359 K 133 046 M Flow usage after the sanitization process TCP UDP TCP used UDP used ISP_Core 159 444 K 127 930 K 42 521 K 55 182 K ISP_Mob 3 904 K 2 063 K 3 454 K 1 850 K 74 / 47

  82. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides Other Improvement Backup Slides Traffic mix by flow in the evaluation dataset Protocol TCP Core TCP Mob UDP Core UDP Mob Generic 12.57% 10.02% 14.16% 13.22% P2P 5.98% 2.57% 13.81% 13.21% Gaming 0.02% 0.00% 0.03% 0.08% Tunnel 10.66% 9.29% 0.02% 0.30% VoIP 0.84% 0.07% 1.94% 1.05% IM 10.60% 0.46% 0.35% 0.05% Streaming 1.07% 0.71% 58.33% 0.42% Mail 14.17% 1.50% 0.00% 0.00% Management 0.01% 0.01% 11.35% 69.93% Filetransfer 0.69% 0.30% 0.00% 0.00% Web 42.98% 74.69% 0.00% 0.00% Other 0.41% 0.38% 0.01% 1.74% 75 / 47

  83. Introduction Backup Slides The Deployment Problem Backup Slides The Maintenance Problem Backup Slides Other Improvement Backup Slides Other Improvement Backup Slides Blocking scenario behavior. Both sides send data, but since there is no acknowledgment from the other side they try to retransmit (after a time which grows in every attempt) until finally break the connection with a RST 76 / 47

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Combining Machine and Automata Learning for Network Traffic Classification Zeynab Sabahi, Fatemeh

Combining Machine and Automata Learning for Network Traffic Classification Zeynab Sabahi, Fatemeh

Combining Machine and Automata Learning for Network Traffic Classification Zeynab Sabahi, Fatemeh Ghassemi, and Zahra Alimadadi TTCS 2020,Tehran, Iran 1 Network Traffic Classification, What & Why?

655 views • 44 slides
Traffic Classification in the Fog Scott E. Coull February 23, 2006 Overview What is traffic

Traffic Classification in the Fog Scott E. Coull February 23, 2006 Overview What is traffic

Traffic Classification in the Fog Scott E. Coull February 23, 2006 Overview What is traffic classification? Communities of Interest for classification BLINC Profiling Internet Backbone Traffic What is missing here? Traffic

965 views • 81 slides
Detection and Classification of Anomalies in Network Traffic Using Generalized Entropies and

Detection and Classification of Anomalies in Network Traffic Using Generalized Entropies and

Introduction Statement problem Mathematical background Algorithm Experiments Conclusions Detection and Classification of Anomalies in Network Traffic Using Generalized Entropies and OC-SVM with Mahalanobis Kernel Jayro Santiago-Paz, Deni

583 views • 23 slides
Need for Classification Classification required To isolate traffic of interest

Need for Classification Classification required To isolate traffic of interest

Need for Classification Classification required To isolate traffic of interest Classification of Internet Traffic To treat special types of traffic in a different manner Some types of classification already seen in Alok

371 views • 9 slides
A Semi-supervised Stacked Autoencoder Approach for Network Traffic Classification Ons Aouedi,

A Semi-supervised Stacked Autoencoder Approach for Network Traffic Classification Ons Aouedi,

A Semi-supervised Stacked Autoencoder Approach for Network Traffic Classification Ons Aouedi, Kandaraj Piamrat, Dhruvjyoti Bagadthey HDR-Nets 2020 Workshop The 28th IEEE International Conference on Network Protocols October 10, 2020 1/29

783 views • 29 slides
Network traffic classification: From theory to practice Pere Barlet-Ros Associate Professor at

Network traffic classification: From theory to practice Pere Barlet-Ros Associate Professor at

Network traffic classification: From theory to practice Pere Barlet-Ros Associate Professor at UPC BarcelonaTech Co-founder and Chairman at Polygraph.io Joint work with: Valentn Carela-Espaol, Tomasz Bujlow and Josep Sol-Pareta

1.67k views • 39 slides
On the challenges of network traffic classification with NetFlow/IPFIX Pere Barlet-Ros Associate

On the challenges of network traffic classification with NetFlow/IPFIX Pere Barlet-Ros Associate

On the challenges of network traffic classification with NetFlow/IPFIX Pere Barlet-Ros Associate Professor at UPC BarcelonaTech (pbarlet@ac.upc.edu) Joint work with: Valentn Carela-Espaol, Tomasz Bujlow and Josep Sol-Pareta This project

544 views • 26 slides
How is SUNET really used? Results of traffic classification on backbone data Wolfgang John and

How is SUNET really used? Results of traffic classification on backbone data Wolfgang John and

MonNet a project for network and traffic monitoring How is SUNET really used? Results of traffic classification on backbone data Wolfgang John and Sven Tafvelin Dept. of Computer Science and Engineering Chalmers University of Technology

760 views • 21 slides
Rendezvous-based Traffic Rendezvous-based Traffic Classification, Measurement, Classification,

Rendezvous-based Traffic Rendezvous-based Traffic Classification, Measurement, Classification,

Rendezvous-based Traffic Rendezvous-based Traffic Classification, Measurement, Classification, Measurement, and Analysis and Analysis ISC/CAIDA Data Collaboration Workshop October 22, 2012 David Plonka & Paul Barford

1.33k views • 26 slides
EPL606 Quality of Service and Traffic Classification 1 Multimedia, Quality of Service: What is

EPL606 Quality of Service and Traffic Classification 1 Multimedia, Quality of Service: What is

EPL606 Quality of Service and Traffic Classification 1 Multimedia, Quality of Service: What is it? Multimedia applications: network audio and video (continuous media) QoS network provides application with level of performance needed

1.57k views • 93 slides
Computer Networks II Prof. Giorgio Ventre a.a. 2009/2010 Network Traffic Classification Alberto

Computer Networks II Prof. Giorgio Ventre a.a. 2009/2010 Network Traffic Classification Alberto

Computer Networks II Prof. Giorgio Ventre a.a. 2009/2010 Network Traffic Classification Alberto Dainotti alberto@unina.it Dipartimento di Informatica e Sistemistica COMICS Research Group Outline Introduction Motivations Why is it

624 views • 15 slides
Traffic Classifier Real Time Classification and Analysis of T1/E1 Traffic 818 West Diamond Avenue

Traffic Classifier Real Time Classification and Analysis of T1/E1 Traffic 818 West Diamond Avenue

Traffic Classifier Real Time Classification and Analysis of T1/E1 Traffic 818 West Diamond Avenue - Third Floor, Gaithersburg, MD 20878 Phone: (301) 670-4784 Fax: (301) 670-9187 Email: info@gl.com Website: http://www.gl.com 1 1 Traffic

228 views • 11 slides
Pentest Accountability By Analyzing Network Traffic & Network Traffic Metadata RP1

Pentest Accountability By Analyzing Network Traffic & Network Traffic Metadata RP1

Pentest Accountability By Analyzing Network Traffic & Network Traffic Metadata RP1 Presentation By Henk van Doorn & Marko Spithoff Relevance Security Audits Company Detects (Attempted) Breach Accountability Of Actions 2

337 views • 22 slides
Early online classification of encrypted traffic streams using multi-fractal features Erik

Early online classification of encrypted traffic streams using multi-fractal features Erik

Early online classification of encrypted traffic streams using multi-fractal features Erik Arestrm, Linkping University Niklas Carlsson, Linkping University Motivation and problem Early flow classification is important for network

629 views • 40 slides
Traffic Classification Rotsos Charalampos , Jurgen Van Gael, Andrew W. Moore, Zoubin Ghahramani

Traffic Classification Rotsos Charalampos , Jurgen Van Gael, Andrew W. Moore, Zoubin Ghahramani

Probabilistic Graphical Models for Semi-Supervised Traffic Classification Rotsos Charalampos , Jurgen Van Gael, Andrew W. Moore, Zoubin Ghahramani Computer Laboratory and Engineering Department, University of Cambridge Traffic classification

257 views • 21 slides
Towards Reliable Traffic Classification Using Visual Motifs Wilson Lian 1 John McHugh 1 , 2 Fabian

Towards Reliable Traffic Classification Using Visual Motifs Wilson Lian 1 John McHugh 1 , 2 Fabian

Background Visual Motifs Traffic Classification Evaluation Towards Reliable Traffic Classification Using Visual Motifs Wilson Lian 1 John McHugh 1 , 2 Fabian Monrose 1 1 University of North Carolina at Chapel Hill 2 RedJack, LLC FloCon 2010

652 views • 47 slides
Labeling the Active Route in Interactive Navigational Maps Benjamin Morgan University of

Labeling the Active Route in Interactive Navigational Maps Benjamin Morgan University of

Labeling the Active Route in Interactive Navigational Maps Benjamin Morgan University of Wrzburg 21. December 2012 1 / 40 Outline of the presentation 1. Introduction 2. The goal: active route street-labeling 3. Related work: some

1.08k views • 66 slides
Better Machine Learning Through Data Sa Saleema ema Amershi shi Machine T eaching Group

Better Machine Learning Through Data Sa Saleema ema Amershi shi Machine T eaching Group

Better Machine Learning Through Data Sa Saleema ema Amershi shi Machine T eaching Group Microsoft Research August 14, 2016 Making better sense of data. Better data makes better machine learning. Data + Algorithm = Model Data Algorithm

1.16k views • 81 slides
Chapter 1. Introduction to Graph Theory (Chapters 1.1, 1.31.6, Appendices A.2A.3) Prof.

Chapter 1. Introduction to Graph Theory (Chapters 1.1, 1.31.6, Appendices A.2A.3) Prof.

Chapter 1. Introduction to Graph Theory (Chapters 1.1, 1.31.6, Appendices A.2A.3) Prof. Tesler Math 154 Winter 2020 Prof. Tesler Ch. 1. Intro to Graph Theory Math 154 / Winter 2020 1 / 42 Related courses Math 184: Enumerative

719 views • 42 slides
MPLS Special Purpose Labels

MPLS Special Purpose Labels

MPLS Special Purpose Labels dra2-kompella-mpls-special-purpose-labels-01 Loa Andersson Adrian Farrel Kiree< Kompella Two Things Making the

485 views • 10 slides
CS3102 Theory of Computation www.cs.virginia.edu/~njb2b/cstheory/s2020 Warm up: There are two

CS3102 Theory of Computation www.cs.virginia.edu/~njb2b/cstheory/s2020 Warm up: There are two

CS3102 Theory of Computation www.cs.virginia.edu/~njb2b/cstheory/s2020 Warm up: There are two categories of computing: 1) Hardware (e.g. CPU) 2) Software (e.g. Java) Are they different? How are they the same? Differences Hardware (CPU)

769 views • 27 slides
The AIM Problem in Loop Theory Conjecture: Let Q be an Abelian Inner Mapping (AIM) loop. Then Q/N

The AIM Problem in Loop Theory Conjecture: Let Q be an Abelian Inner Mapping (AIM) loop. Then Q/N

The AIM Problem in Loop Theory Conjecture: Let Q be an Abelian Inner Mapping (AIM) loop. Then Q/N ( Q ) is an abelian group and Q/Z ( Q ) is a group. In particular, Q is nilpotent of class at most 3. M. Kinyon. cl-informatik.uibk.ac.at/users/cek/

538 views • 10 slides
Semantic Labelling for Proving Termination of Combinatory Reduction Systems Makoto Hamana

Semantic Labelling for Proving Termination of Combinatory Reduction Systems Makoto Hamana

Semantic Labelling for Proving Termination of Combinatory Reduction Systems Makoto Hamana Department of Computer Science, Gunma University, Japan WFLP09 June, 2009. 1 Intro: Termination Proof by RPO Term Rewriting System (TRS) R : fact

540 views • 17 slides
SUSY gauge theories SUSY QCD Consider a SUSY SU ( N ) with F flavors of quarks and

SUSY gauge theories SUSY QCD Consider a SUSY SU ( N ) with F flavors of quarks and

SUSY gauge theories SUSY QCD Consider a SUSY SU ( N ) with F flavors of quarks and squarks Q i = ( i , Q i , F i ) , i = 1 , . . . , F , where is the squark and Q is the quark. Q i = ( i , Q i , F i ) , in the antifundamental

406 views • 40 slides

More recommend