Bro 2.0 and Beyond Network Attack Detection and Defense Early - - PowerPoint PPT Presentation

▶
bro 2 0 and beyond
SMART_READER_LITE
LIVE PREVIEW

Bro 2.0 and Beyond Network Attack Detection and Defense Early - - PowerPoint PPT Presentation

Aug 29, 2022 111 likes •743 views

The Bro Network Security Monitor Bro 2.0 and Beyond Network Attack Detection and Defense Early Warning Systems Schloss Dagstuhl, 2012 Dagstuhl 2012 2 Outline Bro Introduction Much different from the typical IDS you may know Hot off

slide-1
SLIDE 1

Dagstuhl 2012

Network Attack Detection and Defense Early Warning Systems Schloss Dagstuhl, 2012

Bro 2.0 and Beyond

The Bro Network Security Monitor

slide-2
SLIDE 2

2

slide-3
SLIDE 3

Dagstuhl 2012

Bro Introduction

“Much different from the typical IDS you may know”

Hot off the Press: Bro 2.0

Focus on operational deployment

Current Research Projects

Real-time Intelligence Performance for next-gen environments

3

Outline

slide-4
SLIDE 4

Dagstuhl 2012

4

What is Bro?

slide-5
SLIDE 5

Dagstuhl 2012

Packet Capture

4

What is Bro?

slide-6
SLIDE 6

Dagstuhl 2012

Packet Capture Traffic Inspection

4

What is Bro?

slide-7
SLIDE 7

Dagstuhl 2012

Packet Capture Traffic Inspection Attack Detection

4

What is Bro?

slide-8
SLIDE 8

Dagstuhl 2012

Packet Capture Traffic Inspection Attack Detection Log Recording

NetFlow syslog

4

What is Bro?

slide-9
SLIDE 9

Dagstuhl 2012

Packet Capture Traffic Inspection Attack Detection

Flexibility Abstraction Data Structures

Log Recording

NetFlow syslog

4

What is Bro?

slide-10
SLIDE 10

Dagstuhl 2012

Packet Capture Traffic Inspection Attack Detection

Flexibility Abstraction Data Structures

Log Recording

NetFlow syslog

4

What is Bro?

slide-11
SLIDE 11

Dagstuhl 2012

Packet Capture Traffic Inspection Attack Detection

Flexibility Abstraction Data Structures

Log Recording

NetFlow syslog

Flexibility Abstraction Data Structures

4

What is Bro?

slide-12
SLIDE 12

Dagstuhl 2012

Packet Capture Traffic Inspection Attack Detection

Flexibility Abstraction Data Structures

Log Recording

“Domain-specific Python”

NetFlow syslog

Flexibility Abstraction Data Structures

4

What is Bro?

slide-13
SLIDE 13

Dagstuhl 2012

Fundamentally different from other IDS.

Reset your idea of an IDS before starting to use Bro.

Real-time network analysis framework.

Primarily an IDS, but many use it for general traffic analysis.

Can accommodate a range of detection approaches.

Policy-neutral at the core.

Highly stateful.

Tracks extensive application-layer network state.

Supports forensics.

Extensively logs what it sees.

5

Philosophy

slide-14
SLIDE 14

Dagstuhl 2012

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

6

Bro History

1995 2010 1996

Vern writes 1st line of code

slide-15
SLIDE 15

Dagstuhl 2012

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

6

Bro History

1995 2010 1996

Vern writes 1st line of code LBNL starts using Bro

  • perationally
slide-16
SLIDE 16

Dagstuhl 2012

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

6

Bro History

1995 2010 1996

Vern writes 1st line of code Bro Waters Bro 2.0 v0.2 1st CHANGES entry v0.6 RegExps Login analysis v0.8aX/0.9aX SSL/SMB

STABLE releases

BroLite v1.1/v1.2 when Stmt Resource tuning Broccoli DPD v1.5 BroControl v0.7a90 Profiling State Mgmt v1.4 DHCP/BitTorrent HTTP entities NetFlow Bro Lite Deprecated v1.0 BinPAC IRC/RPC analyzers 64-bit support Sane version numbers v0.4 HTTP analysis Scan detector IP fragments Linux support v0.7a175/0.8aX Signatures SMTP IPv6 support User manual v0.7a48 Consistent CHANGES v1.3 Ctor expressions GeoIP Conn Compressor 0.8a37 Communication Persistence Namespaces Log Rotation LBNL starts using Bro

  • perationally
slide-17
SLIDE 17

Dagstuhl 2012

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2011

6

Bro History

1995

USENIX Paper Stepping Stone Detector Anonymizer Active Mapping Context Signat. TRW State Mgmt.

  • Independ. State

Host Context Time Machine Enterprise Traffic BinPAC DPD 2nd Path Bro Cluster Shunt Autotuning Parallel Prototype

2010 1996

Vern writes 1st line of code Bro Waters Bro 2.0 v0.2 1st CHANGES entry v0.6 RegExps Login analysis v0.8aX/0.9aX SSL/SMB

STABLE releases

BroLite v1.1/v1.2 when Stmt Resource tuning Broccoli DPD v1.5 BroControl v0.7a90 Profiling State Mgmt v1.4 DHCP/BitTorrent HTTP entities NetFlow Bro Lite Deprecated v1.0 BinPAC IRC/RPC analyzers 64-bit support Sane version numbers v0.4 HTTP analysis Scan detector IP fragments Linux support v0.7a175/0.8aX Signatures SMTP IPv6 support User manual v0.7a48 Consistent CHANGES v1.3 Ctor expressions GeoIP Conn Compressor 0.8a37 Communication Persistence Namespaces Log Rotation LBNL starts using Bro

  • perationally

Academic Publications

slide-18
SLIDE 18

Dagstuhl 2012

“Who’s Using It?”

7

slide-19
SLIDE 19

Dagstuhl 2012

8

Example Logs

slide-20
SLIDE 20

Dagstuhl 2012

> bro -i en0 [ ... wait ...] > cat conn.log

8

Example Logs

slide-21
SLIDE 21

Dagstuhl 2012

> bro -i en0 [ ... wait ...] > cat conn.log

#fields ts id.orig_h id.orig_p id.resp_h id.resp_p proto service duration

1144876741.1198 192.150.186.169 53115 82.94.237.218 80 tcp http 16.14929 1144876612.6063 192.150.186.169 53090 198.189.255.82 80 tcp http 4.437460 1144876596.5597 192.150.186.169 53051 193.203.227.129 80 tcp http 0.372440 1144876606.7789 192.150.186.169 53082 198.189.255.73 80 tcp http 0.597711 1144876741.4693 192.150.186.169 53116 82.94.237.218 80 tcp http 16.02667 1144876745.6102 192.150.186.169 53117 66.102.7.99 80 tcp http 1.004346 1144876605.6847 192.150.186.169 53075 207.151.118.143 80 tcp http 0.029663

8

Example Logs

slide-22
SLIDE 22

Dagstuhl 2012

> bro -i en0 [ ... wait ...] > cat conn.log > cat http.log

#fields ts id.orig_h id.orig_p id.resp_h id.resp_p proto service duration

1144876741.1198 192.150.186.169 53115 82.94.237.218 80 tcp http 16.14929 1144876612.6063 192.150.186.169 53090 198.189.255.82 80 tcp http 4.437460 1144876596.5597 192.150.186.169 53051 193.203.227.129 80 tcp http 0.372440 1144876606.7789 192.150.186.169 53082 198.189.255.73 80 tcp http 0.597711 1144876741.4693 192.150.186.169 53116 82.94.237.218 80 tcp http 16.02667 1144876745.6102 192.150.186.169 53117 66.102.7.99 80 tcp http 1.004346 1144876605.6847 192.150.186.169 53075 207.151.118.143 80 tcp http 0.029663

8

Example Logs

slide-23
SLIDE 23

Dagstuhl 2012

> bro -i en0 [ ... wait ...] > cat conn.log

#fields ts id.orig_h id.orig_p [...] host uri status_code user_agent [...] 1144876741.6335 192.150.186.169 53116 docs.python.org /lib/lib.css 200 Mozilla/5.0 1144876742.1687 192.150.186.169 53116 docs.python.org /icons/previous.png 304 Mozilla/5.0 1144876741.2838 192.150.186.169 53115 docs.python.org /lib/lib.html 200 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/up.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/next.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/contents.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/modules.png 304 Mozilla/5.0 1144876742.3338 192.150.186.169 53116 docs.python.org /icons/index.png 304 Mozilla/5.0 1144876745.6144 192.150.186.169 53117 www.google.com / 200 Mozilla/5.0

> cat http.log

#fields ts id.orig_h id.orig_p id.resp_h id.resp_p proto service duration

1144876741.1198 192.150.186.169 53115 82.94.237.218 80 tcp http 16.14929 1144876612.6063 192.150.186.169 53090 198.189.255.82 80 tcp http 4.437460 1144876596.5597 192.150.186.169 53051 193.203.227.129 80 tcp http 0.372440 1144876606.7789 192.150.186.169 53082 198.189.255.73 80 tcp http 0.597711 1144876741.4693 192.150.186.169 53116 82.94.237.218 80 tcp http 16.02667 1144876745.6102 192.150.186.169 53117 66.102.7.99 80 tcp http 1.004346 1144876605.6847 192.150.186.169 53075 207.151.118.143 80 tcp http 0.029663

8

Example Logs

slide-24
SLIDE 24

Dagstuhl 2012

> bro -i en0 [ ... wait ...] > cat conn.log

#fields ts id.orig_h id.orig_p [...] host uri status_code user_agent [...] 1144876741.6335 192.150.186.169 53116 docs.python.org /lib/lib.css 200 Mozilla/5.0 1144876742.1687 192.150.186.169 53116 docs.python.org /icons/previous.png 304 Mozilla/5.0 1144876741.2838 192.150.186.169 53115 docs.python.org /lib/lib.html 200 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/up.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/next.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/contents.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/modules.png 304 Mozilla/5.0 1144876742.3338 192.150.186.169 53116 docs.python.org /icons/index.png 304 Mozilla/5.0 1144876745.6144 192.150.186.169 53117 www.google.com / 200 Mozilla/5.0

> cat http.log

#fields ts id.orig_h id.orig_p id.resp_h id.resp_p proto service duration

1144876741.1198 192.150.186.169 53115 82.94.237.218 80 tcp http 16.14929 1144876612.6063 192.150.186.169 53090 198.189.255.82 80 tcp http 4.437460 1144876596.5597 192.150.186.169 53051 193.203.227.129 80 tcp http 0.372440 1144876606.7789 192.150.186.169 53082 198.189.255.73 80 tcp http 0.597711 1144876741.4693 192.150.186.169 53116 82.94.237.218 80 tcp http 16.02667 1144876745.6102 192.150.186.169 53117 66.102.7.99 80 tcp http 1.004346 1144876605.6847 192.150.186.169 53075 207.151.118.143 80 tcp http 0.029663

8

Example Logs

#fields ts id.orig_h id.orig_p [...] host uri status_code user_agent [...] 1144876741.6335 192.150.186.169 53116 docs.python.org /lib/lib.css 200 Mozilla/5.0 1144876742.1687 192.150.186.169 53116 docs.python.org /icons/previous.png 304 Mozilla/5.0 1144876741.2838 192.150.186.169 53115 docs.python.org /lib/lib.html 200 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/up.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/next.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/contents.png 304 Mozilla/5.0 1144876742.3337 192.150.186.169 53116 docs.python.org /icons/modules.png 304 Mozilla/5.0 1144876742.3338 192.150.186.169 53116 docs.python.org /icons/index.png 304 Mozilla/5.0 1144876745.6144 192.150.186.169 53117 www.google.com / 200 Mozilla/5.0

slide-25
SLIDE 25

Dagstuhl 2012

Task: Report all Web requests for files called “passwd”.

9

Script Example: Matching URLs

slide-26
SLIDE 26

Dagstuhl 2012

event http_request(c: connection, # Connection. method: string, # HTTP method.

  • riginal_URI: string, # Requested URL.

unescaped_URI: string, # Decoded URL. version: string) # HTTP version. { if ( method == "GET" && unescaped_URI == /.*passwd/ ) NOTICE(...); # Alarm. }

Task: Report all Web requests for files called “passwd”.

9

Script Example: Matching URLs

slide-27
SLIDE 27

Dagstuhl 2012

Task: Count failed connection attempts per source address.

10

Script Example: Scan Detector

slide-28
SLIDE 28

Dagstuhl 2012

global attempts: table[addr] of count &default=0; event connection_rejected(c: connection) { local source = c$id$orig_h; # Get source address. local n = ++attempts[source]; # Increase counter. if ( n == SOME_THRESHOLD ) # Check for threshold. NOTICE(...); # Alarm. }

Task: Count failed connection attempts per source address.

10

Script Example: Scan Detector

slide-29
SLIDE 29

Dagstuhl 2012

11

Distributed Scripts

slide-30
SLIDE 30

Dagstuhl 2012

Bro comes with >10,000 lines of script code.

Prewritten functionality that’s just loaded.

Scripts also generate the logs.

Amendable to extensive customization and extension.

11

Distributed Scripts

slide-31
SLIDE 31

Dagstuhl 2012

Version 2.0

12

slide-32
SLIDE 32

Dagstuhl 2012

Version 2.0

12

Default scripts rewritten from scratch. Focus ease of use and operational deployment. New logging infrastructure. New build and packaging system. New auto-documentation system (Broxygen). Lots of bugs fixed. Obsolete code removed. New development infrastructure. New regression testing framework. New web server. New mailing lists. New logo.

slide-33
SLIDE 33

Dagstuhl 2012

Upcoming

13

slide-34
SLIDE 34

Dagstuhl 2012

Upcoming

Bro 2.1

Overhauled IPv6 support. New user’s guide. Logging extensions.

Binary logging/Postgresql/CouchDB/SQLite(?) / Threads.

Input framework. Reaction framework. New/improved analyzers.

Syslog/GridFTP/NFS/SMB/BitTorrent.

Extended test-suite.

13

Aiming for 3-4 months release cycle.

slide-35
SLIDE 35

Dagstuhl 2012

Current Research Real-Time Intelligence

14

slide-36
SLIDE 36

Dagstuhl 2012

REN-ISAC’s Security Event System

15

Source: REN-ISAC

slide-37
SLIDE 37

Dagstuhl 2012

Argonne Federated Model

16

Source: Argonne National Lab

slide-38
SLIDE 38

Dagstuhl 2012

Real-time Intelligence with Bro

17

slide-39
SLIDE 39

Dagstuhl 2012

Real-time Intelligence with Bro

17

Bro Policy Script Output Framework

ASCII Binary DBs Python

slide-40
SLIDE 40

Dagstuhl 2012

Real-time Intelligence with Bro

17

Bro Policy Script Output Framework

ASCII

External Partners

Binary DBs Python

slide-41
SLIDE 41

Dagstuhl 2012

Real-time Intelligence with Bro

17

Bro Policy Script Output Framework

ASCII

Input Framework External Partners

ASCII Binary DBs Binary DBs Python Python

slide-42
SLIDE 42

Dagstuhl 2012

Real-time Intelligence with Bro

17

Bro Policy Script Output Framework

ASCII

Input Framework External Partners

ASCII Binary DBs Binary DBs Python Python

Research Questions

What capabilities does the new context give us? What is the quality of the shared information? Do sites see the similar attacks?

slide-43
SLIDE 43

Dagstuhl 2012

Current Research Performance

18

slide-44
SLIDE 44

Dagstuhl 2012

Back in 2005 ...

19

slide-45
SLIDE 45

Dagstuhl 2012

Back in 2005 ...

19

Data: Leibniz-Rechenzentrum, München

20 40 60 80 TBytes/month 1997 1998 1999 2000 2001 2002 2003 2004 2005 Total bytes Incoming bytes

Total upstream bytes Incoming bytes Munich Scientific Network (Today)

3 major universities, 2x10GE upstream ~100,000 Users ~65,000 Hosts

slide-46
SLIDE 46

Dagstuhl 2012

200 400 600 800 TBytes/month 1996 1998 2000 2002 2004 2006 2008 2010 Total bytes Incoming bytes Oct 2005

Today ...

20

Total upstream bytes Incoming bytes

Data: Leibniz-Rechenzentrum, München

Munich Scientific Network

3 major universities, 2x10GE upstream ~100,000 Users ~65,000 Hosts

slide-47
SLIDE 47

Dagstuhl 2012

Load-balancing Architecture

21

slide-48
SLIDE 48

Dagstuhl 2012

Load-balancing Architecture

21

Detection Logic Packet Analysis

NIDS

10Gbps

slide-49
SLIDE 49

Dagstuhl 2012

Load-balancing Architecture

21

10Gbps

External Packet Load-Balancer

Flows

Detection Logic Packet Analysis

NIDS 2

Detection Logic Packet Analysis

NIDS 1

Detection Logic Packet Analysis

NIDS 3

slide-50
SLIDE 50

Dagstuhl 2012

Load-balancing Architecture

21

10Gbps

External Packet Load-Balancer

Flows

Detection Logic Packet Analysis

NIDS 2

Detection Logic Packet Analysis

NIDS 1

Detection Logic Packet Analysis

NIDS 3

Communication Communication

slide-51
SLIDE 51

Dagstuhl 2012

Load-balancing Architecture

21

10Gbps

External Packet Load-Balancer

Flows

“ B r

  • C

l u s t e r ”

Detection Logic Packet Analysis

NIDS 2

Detection Logic Packet Analysis

NIDS 1

Detection Logic Packet Analysis

NIDS 3

Communication Communication

slide-52
SLIDE 52

Dagstuhl 2012

cPacket’s cFlow 10G

22

slide-53
SLIDE 53

Dagstuhl 2012

cPacket’s cFlow 10G

22

slide-54
SLIDE 54

Dagstuhl 2012

Next Stop: 100 Gb/s

23

Source: ESNet

Now these sites need a monitoring solution ... Working with cPacket on a 100GE load- balancer!

Source: ESNet

DOE/ESNet 100G Advanced Networking Initiative

slide-55
SLIDE 55

Dagstuhl 2012

100 Gb/s Load-balancer

slide-56
SLIDE 56

Dagstuhl 2012

100 Gb/s Load-balancer

100Gbps

cFlow 100G

10Gb/s

Bro Cluster

slide-57
SLIDE 57

Dagstuhl 2012

100 Gb/s Load-balancer

100Gbps

cFlow 100G

10Gb/s

Bro Cluster

API

Control

slide-58
SLIDE 58

Dagstuhl 2012

Host Application

Application Core

C Interface Stubs

Native Executable Analysis Specification Native Object Code System Linker Analysis Compiler HILTI Machine Code HILTI Compiler Runtime Library

HILTI Machine Environment OS Toolchain

Improving Bro’s Performance

25

A High-Level Intermediary Language for Traffic Inspection

slide-59
SLIDE 59

Dagstuhl 2012

Host Application

Application Core

C Interface Stubs

Native Executable Analysis Specification Native Object Code System Linker Analysis Compiler HILTI Machine Code HILTI Compiler Runtime Library

HILTI Machine Environment OS Toolchain

Improving Bro’s Performance

Bottlenecks: Script interpretation & single-thread structure

25

A High-Level Intermediary Language for Traffic Inspection

slide-60
SLIDE 60

Dagstuhl 2012

Summary

26

slide-61
SLIDE 61

Dagstuhl 2012

Summary

27

www.bro-ids.org git.bro-ids.org tracker.bro-ids.org @Bro_IDS on Twitter

slide-62
SLIDE 62

Dagstuhl 2012

Summary

Bro 2.0 is a major step forward.

From research to operations. Crucial engineering resources available. Aiming to setup a long-term development model.

27

www.bro-ids.org git.bro-ids.org tracker.bro-ids.org @Bro_IDS on Twitter

slide-63
SLIDE 63

Dagstuhl 2012

Summary

Bro 2.0 is a major step forward.

From research to operations. Crucial engineering resources available. Aiming to setup a long-term development model.

Bro remains a research platform.

Real-time intelligence Performance for next-gen environments

27

www.bro-ids.org git.bro-ids.org tracker.bro-ids.org @Bro_IDS on Twitter