ImprovedDetectionofLow-ProfileProbes andDenial-of-ServiceAttacks * - - PowerPoint PPT Presentation

▶
improved detection of low profile probes
SMART_READER_LITE
LIVE PREVIEW

ImprovedDetectionofLow-ProfileProbes andDenial-of-ServiceAttacks * - - PowerPoint PPT Presentation

Sep 30, 2023 301 likes •511 views

ImprovedDetectionofLow-ProfileProbes andDenial-of-ServiceAttacks * WilliamW.Streilein RobK.Cunningham,SethE.Webster MITLincolnLaboratory WorkshoponStatisticalandMachineLearning

slide-1
SLIDE 1

MITLincolnLaboratory

WWS 6/12/02 1

ImprovedDetectionofLow-ProfileProbes andDenial-of-ServiceAttacks*

WilliamW.Streilein RobK.Cunningham,SethE.Webster MITLincolnLaboratory WorkshoponStatisticalandMachineLearning TechniquesinComputerIntrusionDetection June,2002

*ThisworkwassponsoredbytheDepartmentoftheAirForceunder AirForcecontractF19628-00-C-0002.Opinions,interpretations,

conclusions,andrecommendationsarethoseoftheauthorsandarenotnecessarilyendorsedbytheUnitedStatesGovernment

slide-2
SLIDE 2

MITLincolnLaboratory

WWS 6/12/02 2

Outline

  • MotivationandGoals
  • OverallSystemDesign
  • DataStructureImprovements
  • ThePsplice ConnectionLibrary
  • FeatureVectorImprovements

– Featureelementimportance

  • TestResultson1999DARPAEvaluationData

– Methodology – Results:Probe,DoS,StealthDetection

  • DataStructurePerformance
  • Summary
slide-3
SLIDE 3

MITLincolnLaboratory

WWS 6/12/02 3

Motivation

  • Network-basedintrusiondetectionremainsanimportant

toolindetectingProbesandDoSattacks

– Morecompleteperspectiveonlocalandremotenetwork activitiesthanhost-basedsystems – Canprotectmultipledevicesatonce – Stealthyprobescommonprecursortoattack

  • Stealthyprobesusetechniquesdesignedtoavoiddetection

– DoSattacksincreasinginnumberanddistributednature

  • MachinelearningforProbeandDoSdetection

– Learntodistinguishattacktrafficfromnormaltraffic

  • Actualnetworkdatausedtotrainalgorithm

– Networkoutputsrepresentprobabilitiesofdetection

  • Allowschoosinganappropriateoutputthreshold
slide-4
SLIDE 4

MITLincolnLaboratory

WWS 6/12/02 4

Goals

  • EnhanceProbeandDoSdetectionandperformanceof
  • riginalsystem
  • Enhancedetectionfeaturevector

– Reflectcloserrelationshiptoconnectioneventsandimproved eventtimingavailablefromPsplice

  • DetectnewclassesofAttacks

– ExpandProtocolCoverage:ARP,DHCP – DistributedDoS,Probeattacks

  • Makeinternaldatastructureslessvulnerabletoattack

– Originalsystemuseshashtables

  • Reduceanalystworkloadthroughalertaggregation

– Originalsystemproducesnewalertperconnection

slide-5
SLIDE 5

MITLincolnLaboratory

WWS 6/12/02 5

SystemDesign

  • Multi-stageprocessingof

connectioneventsforattack detection

  • Connectionprocessinghandled

byPsplicelibrary

  • Featureextractionuponevent

timeandcharacteristics

  • Neuralnetworkclassifiers

produceprobabilityofattack

  • Distributedattacksrecognized

throughalertaggregation

  • Aggregationofalertsreduces

analystoverload

AlertAggregation

slide-6
SLIDE 6

MITLincolnLaboratory

WWS 6/12/02 6

Psplice

  • LL-DevelopedlibraryfortrackingTCP,UDP,ICMP,ARPandDHCP

connections

  • Utilizeslibpcappacketlibrary
  • Operatesinreal-timeoroncapturefile
  • SummarizespacketdataintoOPEN,DATAandCLOSEconnection

eventsforcallingapplication

  • Deliversconnectionstatisticsandattributeflags

– ACKflag,FINflag,etc. – Nbytes/pktstosrc,Mbytes/pktstodest

  • Reliableconnectiontrackingalgorithmlessvulnerablethan

traditionaltrackingtechniquestoinsertionanddeletionattacks (c.f.Ptacek,Newsham)

– Waitsforhostresponsetodeterminestateofconnection

slide-7
SLIDE 7

MITLincolnLaboratory

WWS 6/12/02 7

DataStructureEnhancements

  • Supportnewdetectioncapabilityandgeneralfunctionality

– Expandedprotocolcoverage:ARP,DHCP – DistributedDoS,Probeattacks – Alertaggregation

  • BalancedBinaryTreebaseddatastructuresreplacehash

tablesandlinkedlists

– PredictableinsertionandsearchtimesO(logN) – Moreefficientuseofmemory – Lessvulnerabletotargetedattackfromknowledgeable attacker

slide-8
SLIDE 8

MITLincolnLaboratory

WWS 6/12/02 8

NewDataStructures

AnomalyTable

Connectionprobabilities

ARPTable

MAC/IPAddressingMapping

DHCPTable

Networkconfiguration

DoSTable

Store/AggregateDoSAlerts

ProbeTable

Store/AggregateProbeAlerts

AlertTable

Store/AggregateAllAlerts

Determineconnectionlikelihood DetectARPAttacks DetectDHCPAttacks DetectDistributedDoS DetectDistributedProbe ReduceFalseAlarms,Aggregate AlertOutput

DataStructure Functionality

slide-9
SLIDE 9

MITLincolnLaboratory

WWS 6/12/02 9

FeatureVectorElements

SinglePacketFeatures

Protocol(ICMP,TCP,UDP), Strange/InsideIP,FlagsFIN,ACK

ConnectionOpenFeatures

#SameHost,#SameSVC

ConnectionCloseFeatures

#SameHost,#SameSVC,# Abnormal

ConnectionDestinationFeatures

#SameSVC,#DiffSVC

ConnectionSourceFeatures

#DiffPingsfromsource

ConnectionTimingFeatures

OpenInterval,CloseInterval

CaptureIndividualPacket Characteristics,InvalidIPs AbnormalnumberofOPENsmark DoSandfastscans Captureanomalousconnections, Abnormalconnectioncounts CapturetargetedDoS,broad serviceprobe Findactivesingleattacksource DoShavesmallinterconnection eventinterval

Type Intuition/Purpose

slide-10
SLIDE 10

MITLincolnLaboratory

WWS 6/12/02 10

FeatureImportance

  • BackwardfeatureselectionontrainingsetofProbeandDoSattacks
  • PerformancefallsoffafterCLOSEintervaltimingisremoved
  • Bestfeature:“#ofdifferentservicesconnectedto”

ICMP OPEN Interval #ECHOS #OPENS CLOSE Interval STRANGE IP/PORT INSIDE IP #DIFF SVCs

FeatureName

slide-11
SLIDE 11

MITLincolnLaboratory

WWS 6/12/02 11

MostImportantFeatureElements

CaptureIndividualPacket Characteristics,Invalid IPs Abnormalnumber ofOPENsmark

  • fDoSandfastscan

Captureanomalousconnections, Abnormalconnectioncounts CapturetargetedDoS,broad servicescan Findactivesingleattacksource DoSandStealthscanshave smallintereventinterval

Type Reason

SinglePacketFeatures

Protocol(ICMP,TCP,UDP), Strange/InsideIP,FlagsFIN,ACK

ConnectionOpenFeatures

#SameHost,#SameSVC

ConnectionCloseFeatures

#SameHost,#SameSVC,# Abnormal

ConnectionDestinationFeatures

#SameSVC,#DiffSVC

ConnectionSourceFeatures

#DiffPingsfromsource

ConnectionTimingFeatures

OpenInterval,CloseInterval

slide-12
SLIDE 12

MITLincolnLaboratory

WWS 6/12/02 12

DARPAIntrusionDetectionEvaluation DARPAIntrusionDetectionEvaluation SimulationNetworkOverview SimulationNetworkOverview Outside

Internet Eyrie AFBase

  • http
  • smtp
  • pop3
  • FTP
  • IRC
  • Telnet
  • X
  • SQL/Telnet
  • DNS
  • finger
  • snmp
  • time

PrimaryServices/Protocols PacketSniffer

Inside

Router

1000’sHosts,100’sUsers Normal andAttack Traffic

  • UNIXWorkstations
  • CISCORouter
slide-13
SLIDE 13

MITLincolnLaboratory

WWS 6/12/02 13

Training/TestingMethodology

  • 1998and1999DARPAEvaluationCorpus

– Severalweeksofattack-freeandattack-ladennetworkdata

  • Probes,DoS,UsertoRoot,RemotetoLocalattacks
  • Note:DoesNOTcontainDHCPordistributedProbes/DoSattacks

– UsedbyotherresearchersforIDSdevelopment

  • Wellover200downloadstodate,~50citations(citeseer)
  • FocusonstealthyProbeandflow-basedDoSattacks

– Flow-basedattacksexhaustnetwork/computingresources

  • Trainonall1998dataattacks(trainandtest)and1999training

data

– 22Probes,27DoSattacks

  • Teston1999testdata
  • Developindividualattackclassifierstrainedtorecognizeselffrom

normalANDnon-selfattacks

slide-14
SLIDE 14

MITLincolnLaboratory

WWS 6/12/02 14

Results:ProbeDetection

  • TestedonvarietyofIPaddressandportsscans
  • Slightimprovementoveroriginalsystemduetobetter

connectiontrackingandtiming Detectionrateof82% with<1falsealarm/day

FalseAlarmPerEmulatedDay

slide-15
SLIDE 15

MITLincolnLaboratory

WWS 6/12/02 15

Results:Denial-of-ServiceAttacks

  • Testedonsmurf,neptune,PoD
  • SlightimprovementofFArateduetooutputalert

aggregation Detectionrateof68% with<1falsealarm/day

FalseAlarmPerEmulatedDay

slide-16
SLIDE 16

MITLincolnLaboratory

WWS 6/12/02 16

Results:StealthyAttacks

  • Testedonstealthyattacks
  • Significantperformanceimprovementduetobetter

half-OPENconnectiontrackingbyPsplice Detectionrateof100% with.4falsealarm/day

FalseAlarmPerEmulatedDay

slide-17
SLIDE 17

MITLincolnLaboratory

WWS 6/12/02 17

DataStructurePerformanceunder LinearIPAddressScan

  • Commontechniqueoflinearaddress/portscanningcauses

frequentrebalancing:

  • Worstcaseforbinarytree
  • Bestcaseforhashtablegivenmodulus

InsertandSearchTimes MemoryUsage

slide-18
SLIDE 18

MITLincolnLaboratory

WWS 6/12/02 18

Aggregationtechniques

  • Aggregationcanreduceanalystworkload

– AlertsgroupedbyIPsrcandattacktype – Timerusedtosquelchrepeatedalertevents – Only1alertperattack – Allowssummarizationofattack

  • Canreducefalsealarmusingalertcounts

– DoSattacks(Neptune):waitforNalertsbeforefiring

  • SomedatacanlooklikeaDoSattack,butisisolatedintraffic
  • Learn‘N’fromtrainingdata

– Allothers,alertimmediately

  • Portsweep,ipsweep,teardrop,etc.
slide-19
SLIDE 19

MITLincolnLaboratory

WWS 6/12/02 19

Conclusion

  • Network-basedintrusiondetectionsystemhasbeen

enhancedforbetterdetectionperformanceandattack robustness

  • Additionalstructuresprovidenewfunctionality

– Expandedprotocolcoverage:ARP/DHCP – Alertaggregation(Probe,DoSTables)

  • ImprovedDetectionperformance

– Enhancedfeaturevector

  • ExploitscloserrelationshiptonetworkeventsfromPsplice

– Results:Significantlyimproveddetectionofstealthyattacks

  • Balancedbinarytreedatastructuresdemonstrate

robustnessunderworstcasescenariooflinearIPaddress scan

slide-20
SLIDE 20

MITLincolnLaboratory

WWS 6/12/02 20

FutureWork

  • DynamicNetworkProfiling

– Currentsystemusesstaticfileofconnectionprobabilities – Facilitatedeployment

  • Enhanceddistributedattackdetection

– Spoofedsourceprobes,DoSattacksshareotherfeatures

  • Target,durationofattack,typeofattack

– DoSandProbeTablessupportfunctionality

  • IDMEFsupport

– Utilizestandardalertformatforcorrelationsystems