Following the Packets: A Walk Through Bros Internal Processing - - PowerPoint PPT Presentation

▶
following the packets a walk through bro s internal
SMART_READER_LITE
LIVE PREVIEW

Following the Packets: A Walk Through Bros Internal Processing - - PowerPoint PPT Presentation

Mar 24, 2024 445 likes •898 views

Following the Packets: A Walk Through Bros Internal Processing Pipeline Robin Sommer robin@icir.org Corelight, Inc. International Computer Science Institute Lawrence Berkeley National Laboratory Outline Bros Architecture & Data

slide-1
SLIDE 1

Following the Packets: A Walk Through Bro’s Internal Processing Pipeline

Robin Sommer

robin@icir.org

Corelight, Inc. International Computer Science Institute Lawrence Berkeley National Laboratory

slide-2
SLIDE 2

Outline

Bro’s Architecture & Data Flow Components

Protocol & file analysis Log writer & input readers Bro Plugins

slide-3
SLIDE 3

Bro Architecture

Event Engine Script Interpreter Network

Events Packets

slide-4
SLIDE 4

Bro Architecture

Event Engine Script Interpreter Network

Packet Source

Events Packets

slide-5
SLIDE 5

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop

Events Packets

slide-6
SLIDE 6

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

slide-7
SLIDE 7

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection

slide-8
SLIDE 8

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis

slide-9
SLIDE 9

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis

slide-10
SLIDE 10

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine

slide-11
SLIDE 11

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue

slide-12
SLIDE 12

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue

slide-13
SLIDE 13

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Event Handlers

slide-14
SLIDE 14

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue

Functions Modules Statements Expressions Types Values

Event Handlers

slide-15
SLIDE 15

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

slide-16
SLIDE 16

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer

slide-17
SLIDE 17

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer Communic. Process

slide-18
SLIDE 18

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager Input Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer Communic. Process

slide-19
SLIDE 19

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager Input Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer Thread Manager Communic. Process

slide-20
SLIDE 20

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager Input Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer Thread Manager Communic. Process

slide-21
SLIDE 21

Protocol & File Analysis

IP

Example: SSL Session

slide-22
SLIDE 22

Protocol & File Analysis

IP TCP

connection_established()

Example: SSL Session

slide-23
SLIDE 23

Protocol & File Analysis

IP TCP SSL

connection_established() ssl_{client,server}_hello()

Example: SSL Session

slide-24
SLIDE 24

Protocol & File Analysis

IP TCP SSL X.509

connection_established() ssl_{client,server}_hello() x509_certificate()

Example: SSL Session

slide-25
SLIDE 25

Protocol & File Analysis

IP TCP SSL X.509

connection_established() ssl_{client,server}_hello() x509_certificate()

?

Example: SSL Session

slide-26
SLIDE 26

Dynamic Protocol Detection

IP TCP

slide-27
SLIDE 27

Dynamic Protocol Detection

IP TCP PIA

Buffer

slide-28
SLIDE 28

Dynamic Protocol Detection

IP TCP PIA

Buffer

Analyzer::register_for_port(Analyzer::SSL, 443/tcp);

slide-29
SLIDE 29

Dynamic Protocol Detection

IP TCP PIA

Buffer

signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[…].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp);

slide-30
SLIDE 30

Dynamic Protocol Detection

IP TCP SSL PIA

Buffer

signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[…].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp);

slide-31
SLIDE 31

Dynamic Protocol Detection

IP TCP SSL X.509 PIA

Buffer

signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[…].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp);

slide-32
SLIDE 32

Dynamic Protocol Detection

IP TCP SSL X.509 PIA

Buffer

signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[…].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp);

HTTP

slide-33
SLIDE 33

Dynamic Protocol Detection

IP TCP SSL X.509 PIA

Buffer

signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[…].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp);

HTTP

slide-34
SLIDE 34

Protocol Analyzer API

class Analyzer { virtual void Init(); virtual void Done(); virtual void DeliverPacket(int len, const u_char* data, bool orig, bool orig, uint64 seq, const IP_Hdr* ip, int caplen); virtual void DeliverStream(int len, const u_char* data, bool orig); virtual void Undelivered(uint64 seq, int len, bool orig); virtual void EndOfData(bool is_orig); virtual void FlipRoles(); } class TCP_ApplicationAnalyzer : public Analyzer { virtual void EndpointEOF(bool is_orig); virtual void ConnectionFinished(int half_finished); virtual void ConnectionReset(); };

slide-35
SLIDE 35

File Analyzer API

class Analyzer { virtual void Init(); virtual void Done(); virtual bool DeliverChunk(const u_char* data, uint64 len, uint64 offset); virtual bool DeliverStream(const u_char* data, uint64 len); virtual bool EndOfFile(); virtual bool Undelivered(uint64 offset, uint64 len); };

slide-36
SLIDE 36

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager Input Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer Thread Manager Communic. Process

slide-37
SLIDE 37

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager Input Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer Thread Manager Communic. Process

slide-38
SLIDE 38

Writers & Readers

ASCII SQLite

Log Writers Input Readers

ASCII Binary Raw file SQLite

slide-39
SLIDE 39

Log Writer API

Each writer runs in its own thread.

class WriterBackend { virtual bool DoInit(const WriterInfo& info, int num_fields, virtual bool DoWrite(int num_fields, const Field* const* fields, threading::Value** vals); virtual bool DoSetBuf(bool enabled); virtual bool DoFlush(double network_time); virtual bool DoRotate(const char* rotated_path, double open, double close, bool terminating); virtual bool DoFinish(double network_time); virtual bool DoHeartbeat(double network_time, double current_time); };

slide-40
SLIDE 40

Input Reader API

Each reader runs in its own thread.

class ReaderBackend { virtual bool DoInit(const ReaderInfo& info, int arg_num_fields, const threading::Field* const* fields); virtual void DoClose(); virtual bool DoUpdate(); virtual bool DoHeartbeat(double network_time, double current_time); // Simple mode. void SendEvent(const char* name, const int num_vals, threading::Value* *vals); void Put(threading::Value** val); void Delete(threading::Value** val); void Clear(); void EndOfData(); // Tracking mode. void SendEntry(threading::Value** vals); void EndCurrentSend(); };

slide-41
SLIDE 41

Bro Plugins

Log writers Input readers Protocol analyzers
 File analyzers Packet Sources BiF elements

Build & install Bro components independently Distribute as a Bro package

Bro Scripts

slide-42
SLIDE 42

BYOP

# ~/bro/aux/bro-aux/plugin-support/init-plugin icsi-plugin ICSI BroMagic Installing icsi-plugin/CHANGES ... Installing icsi-plugin/CMakeLists.txt ... Installing icsi-plugin/configure ... Installing icsi-plugin/configure.plugin ... Installing icsi-plugin/scripts/__load__.bro ... Installing icsi-plugin/scripts/ICSI/BroMagic/__load__.bro ... Installing icsi-plugin/scripts/init.bro ... Installing icsi-plugin/src/bromagic.bif ... Installing icsi-plugin/src/Plugin.h ... Installing icsi-plugin/src/Plugin.cc … […] # cd icsi-plugin/ # ./configure --brodist=$HOME/bro/master Build Directory : build Bro Source Directory : /home/robin/bro/master […] # make && make install […] # bro -N ICSI::BroMagic - <Insert description> (dynamic, version 0.1) Bro::ARP - ARP Parsing (built-in) Bro::AsciiReader - ASCII input reader (built-in) Bro::AsciiWriter - ASCII log writer (built-in) Bro::AYIYA - AYIYA Analyzer (built-in) […]

slide-43
SLIDE 43

Bro Architecture

Event Engine Script Interpreter Network

Packet Source I/O Loop Session Table

Events Packets

Connection Protocol Analysis File Analysis Signature Engine Event Queue Timer Queue Log Manager Input Manager

Event prototypes Functions Types Constants

Functions Modules Statements Expressions Types Values

Event Handlers

BiF Elements

Remote- Serializer Thread Manager Communic. Process

slide-44
SLIDE 44

Event Engine Script Interpreter Network

Events Packets

Questions?