StreamWorks A System for Real-Time Graph Pattern Matching on - - PowerPoint PPT Presentation

▶
streamworks a system for real time graph pattern matching
SMART_READER_LITE
LIVE PREVIEW

StreamWorks A System for Real-Time Graph Pattern Matching on - - PowerPoint PPT Presentation

May 22, 2023 167 likes •799 views

StreamWorks A System for Real-Time Graph Pattern Matching on Network Traffic GEORGE CHIN, SUTANAY CHOUDHURY AND KHUSHBU AGARWAL Pacific Northwest National Laboratory January 21, 2015 Unclassified 1 Emerging Graph Patterns Goal: Detect

slide-1
SLIDE 1

StreamWorks – A System for Real-Time Graph Pattern Matching on Network Traffic

GEORGE CHIN, SUTANAY CHOUDHURY AND KHUSHBU AGARWAL

January 21, 2015 1

Pacific Northwest National Laboratory

Unclassified

slide-2
SLIDE 2

January 21, 2015 2

Emerging Graph Patterns

Goal:

Detect and identify precursor events and patterns as they emerge in complex networks such that events or threats may be mitigated or acted upon before they are fully realized

Unclassified

Capture evolution of critical graph patterns Devise optimal search strategy to identify emerging pattern Consider cases where target subgraph patterns may or may not be known Subgraph pattern matching is a well-studied NP-hard problem. Some work on scalable algorithms Limited work on subgraph matching in dynamic networks Application areas: Computer network intrusions and threats Social media and network analysis Financial and stock market analysis Distributed sensor networks

slide-3
SLIDE 3

January 21, 2015 3

Emerging Graph Pattern Algorithm in Action

Data Graph

Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

slide-4
SLIDE 4

January 21, 2015 4

Data Graph

Trout (Web Server) Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-5
SLIDE 5

January 21, 2015 5

Data Graph

Trout (Web Server) Goliath (DNS Server) Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-6
SLIDE 6

January 21, 2015 6

Data Graph

Trout (Web Server) Goliath (DNS Server) Pine Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-7
SLIDE 7

January 21, 2015 7

Data Graph

Trout (Web Server) Goliath (DNS Server) Pine Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-8
SLIDE 8

January 21, 2015 8

Data Graph

Trout (Web Server) Goliath (DNS Server) Oak Pine Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-9
SLIDE 9

January 21, 2015 9

Data Graph

Trout (Web Server) Goliath (DNS Server) Oak Pine Cedar Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-10
SLIDE 10

January 21, 2015 10

Data Graph

Trout (Web Server) Goliath (DNS Server) Oak Pine Birch Cedar Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-11
SLIDE 11

January 21, 2015 11

Data Graph

Trout (Web Server) Goliath (DNS Server) Oak Pine Birch Cedar Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-12
SLIDE 12

January 21, 2015 12

Trout (Web Server) Goliath (DNS Server) Oak Pine Birch Cedar

Data Graph

Alder Maple

DNS Server Web Server Host DNS Server Web Server Host DNS Server Web Server Host Host DNS Server Web Server Host

Subgraph Join Tree

DNS Server Web Server Host Host Host

100% 33% 33% 33% 67%

Unclassified

Emerging Graph Pattern Algorithm in Action

slide-13
SLIDE 13

January 21, 2015 13

Detecting Emerging Cyber Attacks

Developing emerging subgraph pattern algorithm in a package we call StreamWorks to detect cyber intrusions and attacks in computer network traffic Constructing set of cyber attack graph patterns related to network scans, reflector attacks, flood attacks, viruses, worms, etc. in collaboration with PNNL cybersecurity analysts Utilizing anonymized internet traces data curated by CAIDA (The Cooperative Association for Internet Data Analysis) at SDSC/UCSD and simulated intrusion detection datasets from the University of New Brunswick’s Information Security Centre of Excellence

Unclassified

slide-14
SLIDE 14

January 21, 2015 14

Witty Worm

Internet worm that began to spread on March 19, 2004 Targeted buffer overflow vulnerability in internet security systems (ISS) products Payload contained phrase “(^.^) insert witty message here (^.^)” Attacked port 4000 with packets of sizes between 796 and 1307

796<=Packet Len<=1307 Port 4000

Host Host Host Host Host Host Host Host Host Host Host Host Host Host Host Host

796<=Packet Len<=1307 Port 4000 796<=Packet Len<=1307 Port 4000 796<=Packet Len<=1307 Port 4000 796<=Packet Len<=1307 Port 4000

Unclassified

slide-15
SLIDE 15

January 21, 2015 15

Distributed Denial-of-Service Smurf Attack

Attacker sends packets to broadcast IP address with spoofed source address of victim’s Packets delivered to intermediate hosts Intermediate hosts reply to return address of victim

Host Host Host ICMP Echo Request ICMP Echo Request ICMP Echo Request ICMP Echo Request ICMP Echo Reply ICMP Echo Reply ICMP Echo Reply Hacker ICMP Echo Request Router Victim

Unclassified

slide-16
SLIDE 16

Distributed Denial-of-Service DNS Amplification Attack

January 21, 2015 16

Agents or zombies generate large number

  • f DNS requests with

spoofed source address DNS servers send 3 different types of responses to victim DNS response packets may be significantly larger than DNS request packets

DNS Query Response | ICMP Dest Unreachable | Frag IP Address DNS Query Response | ICMP Dest Unreachable | Frag IP Address DNS Query Response | ICMP Dest Unreachable | Frag IP Address DNS Query DNS Query DNS Query

DNS Server DNS Server DNS Server Victim Zombie Zombie Zombie

DNS Query DNS Query DNS Query

Unclassified

slide-17
SLIDE 17

January 21, 2015 17

Subgraph Join Tree for DDoS Smurf Attack

Host Host Host ICMP Echo Request ICMP Echo Request ICMP Echo Request ICMP Echo Request ICMP Echo Reply ICMP Echo Reply ICMP Echo Reply Hacker ICMP Echo Request Router Victim

Subgraph Join Tree (Breadth-First) Cyberattack Pattern 100% 14% 43% 43% 86%

ICMP Echo Request

Host Host Host Router

Broadcast Address

ICMP Echo Request ICMP Echo Request

Router Victim

ICMP Echo Request

Host Host Host Victim

ICMP Echo Reply ICMP Echo Reply ICMP Echo Reply

Host Host Host Router Victim

Time < E2 Time < E3 Time < E4 E2 E3 E4

Host Host Host Router Victim

Time < E1 E1 Time < E1 Time < E1

Unclassified

slide-18
SLIDE 18

January 21, 2015 18

DDoS Smurf Attack Query

18

48:06

100% 14% 43% 43% 86%

ICMP Echo Request

Host Host Host Router

Broadcast Address ICMP Echo Request ICMP Echo Request Router Victim

ICMP Echo Request

Host Host Host Victim

ICMP Echo Reply ICMP Echo Reply ICMP Echo Reply Host Host Host Router Victim

Time < E2 Time < E3 Time < E4 E2 E3 E4

Host Host Host Router Victim

Time < E1 E1 Time < E1 Time < E1

Breadth-First SJT

Unclassified

slide-19
SLIDE 19

January 21, 2015 19

DDoS Smurf Attack Query

19

51:39

100% 14% 43% 43% 86%

ICMP Echo Request

Host Host Host Router

Broadcast Address ICMP Echo Request ICMP Echo Request Router Victim

ICMP Echo Request

Host Host Host Victim

ICMP Echo Reply ICMP Echo Reply ICMP Echo Reply Host Host Host Router Victim

Time < E2 Time < E3 Time < E4 E2 E3 E4

Host Host Host Router Victim

Time < E1 E1 Time < E1 Time < E1

Breadth-First SJT

Unclassified

slide-20
SLIDE 20

January 21, 2015 20

DDoS Smurf Attack Query

20

53:11

100% 14% 43% 43% 86%

ICMP Echo Request

Host Host Host Router

Broadcast Address ICMP Echo Request ICMP Echo Request Router Victim

ICMP Echo Request

Host Host Host Victim

ICMP Echo Reply ICMP Echo Reply ICMP Echo Reply Host Host Host Router Victim

Time < E2 Time < E3 Time < E4 E2 E3 E4

Host Host Host Router Victim

Time < E1 E1 Time < E1 Time < E1

Breadth-First SJT

Unclassified

slide-21
SLIDE 21

Scalable Subgraph Pattern Matching Algorithm Semantic Graph Library (SGlib)

Network Analysis Visualizations Stream Processing (Storm)

Run-Time System

January 21, 2015 21

StreamWorks Components

21 Unclassified

Subgraph Join Tree Generation Algorithms

Statistics Collection Tools Graph Pattern Library

Graph Pattern Definition and Join Tree Modeling

slide-22
SLIDE 22

January 21, 2015 22

Scalable Subgraph Matching Algorithm

22 Unclassified

Distributed Implementation of Dynamic Graph Search Update the distributed graph with new edges in parallel Search the updated graph in parallel for unique sub-queries Colors represents unique sub- queries in SJ-Tree Each node in SJ-Tree maintains a match collection Each nodes receive the new set of matches Perform parallel hash join of new and old matches in SJ-Tree at each level

3 2 4 1

slide-23
SLIDE 23

January 21, 2015 23 23 Unclassified

CAIDA Network Traffic (2.49M nodes, 19.55M edges)

Scalability Results for Distributed Implementation of Dynamic Graph Search

Scalable Subgraph Matching Algorithm

PNNL institutional computing (PIC) cluster: 692 nodes, AMD Interlagos processors, dual socket, 16 cores per socket, 64 GB memory per node, QDR InfiniBand

slide-24
SLIDE 24

January 21, 2015 24 24 Unclassified

Distributed Dynamic Graph Data Structure

Scalable Subgraph Matching Algorithm

slide-25
SLIDE 25

January 21, 2015 25 25 Unclassified

Scalable Subgraph Matching Algorithm

Concurrent Graph Queries via Multiple Subgraph Join Trees Conduct parallel searching across all subgraph patterns of all subgraph join trees Leverage locality in terms of operations: Identify common subgraph patterns across subgraph join trees and search

  • nce for multiple queries

Leverage locality in terms of data: Identify graph regions that apply to multiple subgraph searches and track and manage

  • nce for multiple queries
slide-26
SLIDE 26

5 10 15 20 25 30 35 40 1 2 4 8 16 Records Per Second (Thousands) Number of Threads With Filtering/Aggregation Without Filtering/Aggregation

January 21, 2015 26

Stream Processing

26 Unclassified

Developed various Apache Storm Bolts to filter/aggregate Netflow data Filtered and aggregated data is passed to emerging subgraph algorithm using Apache ActiveMQ Tuning of primitive subgraph matching between Storm and emerging subgraph algorithm is ongoing

1M Netflow records through Storm, 1 record per message through ActiveMQ (on PIC)

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2 4 8 16 32 64 Graph Updates Per Second (Millions) Nodes in Cluster BatchSize (10M) BatchSize (100M)

10M and 100M power law graph updates through emerging subgraph algorithm (on PIC)

slide-27
SLIDE 27

January 21, 2015 27

Cyberattack Graph Patterns

27 Unclassified

Port Scan Exploit Spread

Syn Ack Exploit Exploit Exploit

Is Netflow data and patterns enough to effectively detect emerging cyberattacks?

slide-28
SLIDE 28

January 21, 2015 28

Cyberattack Graph Patterns

28 Unclassified

Port Scan Exploit Spread

Syn Ack Exploit Exploit Exploit

New User? Failed Logins? Admin Privileges? Escalated Privileges? New User? Failed Logins? Admin Privileges? Escalated Privileges? Known Host? Type of Server? Machine Type Known to Scan? Newly Added Application? Known Application Creator? Known Process? Known Exploit?

slide-29
SLIDE 29

January 21, 2015 29

Multi-Source Cyberattack Graph Patterns

29 Unclassified

Look to fuse streaming Netflow data with other streaming data sources such event logs, host scan logs, firewall logs, and anti-malware reports Enrich the semantic graph with more attributes to collect information from the fused data stream Derive additional candidate graph patterns and their associated subgraph join trees with fuller attributes to better elaborate specific cyberattacks

slide-30
SLIDE 30

January 21, 2015 30 Unclassified

Automatic Subgraph Join Tree Generation

With known target graph patterns Breadth-first traversal Depth-first traversal Frequency- based With unknown target graph patterns Frequency- based

Host Host Host ICMP Echo Request ICMP Echo Request ICMP Echo Request ICMP Echo Request ICMP Echo Reply ICMP Echo Reply ICMP Echo Reply Hacker ICMP Echo Request Router Victim

slide-31
SLIDE 31

DNS Server A Host 3 Host 2 Host 3 Host 2 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 1 DNS Server A Host 3 Host 2 DNS Server A Web Server B Host 3 DNS Server A Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 2 DNS Server A Web Server B Host 1 Web Server B Host 1 Host 3 Web Server B Host 3 Web Server B Host 3 Host 2 Web Server B Host 1 DNS Server A Host 1 Host 3 Host 2 Web Server B

January 21, 2015 31 31

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-32
SLIDE 32

January 21, 2015 32

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

32

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3 DNS Server A Host 1 Host 3 Host 2 Web Server B

532

DNS Server A Host 3 Host 2

14

Web Server B Host 1 Host 3

27 173 210 182 267 89 96

Host 3 Host 2

756 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 3

1369

slide-33
SLIDE 33

January 21, 2015 33 33

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3 DNS Server A Host 1 Host 3 Host 2 Web Server B

532

DNS Server A Host 3 Host 2

14

Web Server B Host 1 Host 3

27 173 210 182 267 89 96

Host 3 Host 2

756 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 3

1369

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-34
SLIDE 34

January 21, 2015 34 34

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3 DNS Server A Host 1 Host 3 Host 2 Web Server B

532

Web Server B Host 1 Host 3

27 173 210 182 267 89 96

Host 3 Host 2

756 1139 1637 1284

DNS Server A Host 1

1375 1369

DNS Server A Host 3 DNS Server A Host 3 Host 2

14

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-35
SLIDE 35

January 21, 2015 35 35

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532

Web Server B Host 1 Host 3

27 173 210 182 267 89 96

Host 3 Host 2

756 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3 Host 2

14

DNS Server A Host 3

1369

Automatic Join Tree Generation with Known Graph Pattern

slide-36
SLIDE 36

January 21, 2015 36 36

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96

Host 3 Host 2

756 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3

1369

Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Automatic Join Tree Generation with Known Graph Pattern

slide-37
SLIDE 37

January 21, 2015 37 37

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96

Host 3 Host 2

756 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3

1369

Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-38
SLIDE 38

January 21, 2015 38 38

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96

Host 3 Host 2

756 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3

1369

DNS Server A Host 1 Host 3 Host 2 Web Server B Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Automatic Join Tree Generation with Known Graph Pattern

slide-39
SLIDE 39

January 21, 2015 39 39

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B

1369

DNS Server A Host 3 DNS Server A Host 1 Host 3 Host 2 Web Server B Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Host 3 Host 2

756

Automatic Join Tree Generation with Known Graph Pattern

slide-40
SLIDE 40

January 21, 2015 40 40

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3

1369

DNS Server A Host 1 Host 3 Host 2 Web Server B Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Host 3 Host 2

756

DNS Server A Host 1 Host 3 Host 2 Web Server B

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-41
SLIDE 41

January 21, 2015 41 41

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96 1139 1637 1284

DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3

1369

DNS Server A Host 1 Host 3 Host 2 Web Server B Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Host 3 Host 2

756

DNS Server A Host 1 Host 3 Host 2 Web Server B

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-42
SLIDE 42

January 21, 2015 42 42

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96 1139 1637 1284

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3

1369

DNS Server A Host 1 Host 3 Host 2 Web Server B Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Host 3 Host 2

756

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 1

1375

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-43
SLIDE 43

January 21, 2015 43 43

DNS Server A Web Server B Host 3 DNS Server A Web Server B Host 1 DNS Server A Host 1 Host 2 DNS Server A Host 1 Host 3 DNS Server A Host 3 Host 2 DNS Server A Host 3 Host 2 Web Server B Host 3 Host 2 DNS Server A Host 2 Web Server B Host 1 Web Server B Host 3

532 173 210 182 267 89 96 1139 1637 1284

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 3

1369

DNS Server A Host 1 Host 3 Host 2 Web Server B Web Server B Host 1 Host 3

27

DNS Server A Host 3 Host 2

14

Host 3 Host 2

756

DNS Server A Host 1 Host 3 Host 2 Web Server B DNS Server A Host 1

1375

DNS Server A Host 1 Host 3 Host 2 Web Server B

Automatic Frequency-Based Join Tree Generation with Known Graph Pattern

slide-44
SLIDE 44

January 21, 2015 44

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

44

Emergent Infrequent Subgraph Patterns

slide-45
SLIDE 45

January 21, 2015 45

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

45

Web Server J DB Z Host 2

Emergent Infrequent Subgraph Patterns

slide-46
SLIDE 46

January 21, 2015 46

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

46

Web Server J DB Z Host 2 DNS Server A FS Y DB X

Emergent Infrequent Subgraph Patterns

slide-47
SLIDE 47

January 21, 2015 47

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

47

DNS Server G Host 4 DB O Web Server J DB Z Host 2 DNS Server A FS Y DB X

Emergent Infrequent Subgraph Patterns

slide-48
SLIDE 48

January 21, 2015 48

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

48

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X

Emergent Infrequent Subgraph Patterns

slide-49
SLIDE 49

January 21, 2015 49

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

49

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

slide-50
SLIDE 50

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

DNS Server A FS Y DB X Web Server B Host 1 FS Y

January 21, 2015

slide-51
SLIDE 51

January 21, 2015 51

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

51

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X

slide-52
SLIDE 52

January 21, 2015 52

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

52

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B

slide-53
SLIDE 53

January 21, 2015 53

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

53

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X

slide-54
SLIDE 54

January 21, 2015 54

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

54

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B

slide-55
SLIDE 55

January 21, 2015 55

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

55

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B DNS Server C Host 5 FS P

slide-56
SLIDE 56

January 21, 2015 56

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

56

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B DNS Server C Host 5 FS P DNS Server D Web Server E DB Q

slide-57
SLIDE 57

January 21, 2015 57

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

57

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B DNS Server C Host 5 FS P DNS Server D Web Server E DB Q DNS Server A Host 1 FS Y

slide-58
SLIDE 58

January 21, 2015 58

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

58

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B DNS Server C Host 5 FS P DNS Server D Web Server E DB Q DNS Server A Host 1 FS Y DNS Server A Host 1 FS Y

slide-59
SLIDE 59

January 21, 2015 59

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

59

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B DNS Server C Host 5 FS P DNS Server D Web Server E DB Q DNS Server A Host 1 FS Y DNS Server A Host 1 FS Y DNS Server A Host 1 FS Y

slide-60
SLIDE 60

January 21, 2015 60

Automatic Frequency-Based Join Tree Generation with Unknown Graph Pattern

60

DNS Server G Host 4 DB O DNS Server I Host 3 FS N Web Server J DB Z Host 2 DNS Server A FS Y DB X Web Server B Host 1 FS Y

Emergent Infrequent Subgraph Patterns

Web Server B Host 1 FS Y DNS Server A FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B FS Y DB X DNS Server A Host 1 FS Y DB X Web Server B DNS Server C Host 5 FS P DNS Server D Web Server E DB Q DNS Server A Host 1 FS Y DNS Server A Host 1 FS Y DNS Server A Host 1 FS Y DNS Server A Host 1 FS Y DB X Web Server B

slide-61
SLIDE 61

January 21, 2015 61

Summary

Developing scalable emerging subgraph pattern algorithm that can detect and identify precursor events and patterns as they emerge in complex networks Utilizing an efficient and novel subgraph join tree approach which tracks and monitors partial matches of a query graph against a large-scale dynamic network Applying emerging subgraph pattern algorithm to the detection

  • f computer network threats and intrusions

Packaging emerging subgraph pattern capabilities into an interactive network analysis framework called StreamWorks Extending StreamWorks to support emerging subgraph patterns across multiple dynamic data sources Developing approach for dynamic subgraph join tree generation to support the detection of zero-day exploits

Unclassified