Creating a dedicated log management layer Peter Czanik / syslog-ng, - - PowerPoint PPT Presentation

▶
creating a dedicated log management layer
SMART_READER_LITE
LIVE PREVIEW

Creating a dedicated log management layer Peter Czanik / syslog-ng, - - PowerPoint PPT Presentation

Feb 19, 2023 540 likes •907 views

Creating a dedicated log management layer Peter Czanik / syslog-ng, a One Identity business About me Peter Czanik from Hungary Evangelist at One Identity: syslog-ng upstream syslog-ng packaging, support, advocacy syslog-ng originally

slide-1
SLIDE 1

Creating a dedicated log management layer

Peter Czanik / syslog-ng, a One Identity business

slide-2
SLIDE 2

One Identity - Restricted

2

About me

■ Peter Czanik from Hungary ■ Evangelist at One Identity: syslog-ng upstream ■ syslog-ng packaging, support, advocacy syslog-ng originally developed by Balabit, now part of One Identity

slide-3
SLIDE 3

One Identity - Restricted

3

Overview

■ Basics: central log collection ■ Growing complexity: analytics for security & operations ■ Reducing complexity: dedicated log management ■ Implementation using syslog-ng

slide-4
SLIDE 4

#GetIAMRight | One Identity - Restricted - Confjdential

4

Back to basics

■ Central log collection

slide-5
SLIDE 5

One Identity - Restricted

5

Why central logging?

Ease of use One place to check

instead of many

Availability

Even if the sender machine is down

Security

Logs are available even if sender machine is compromised

slide-6
SLIDE 6

#GetIAMRight | One Identity - Restricted - Confjdential

6

Growing and reducing complexity

■ Multiple analytics systems ■ Wasting of resources ■ Consolidating using a unifjed log management layer

slide-7
SLIDE 7

One Identity - Restricted

7

Multiple analytics systems

■ Security, developers, operators use different analytics ■ All come with log aggregation tools ■ Some examples:

 Elastic: Beats and Logstash  Splunk: forwarders  LaaS: collectors

slide-8
SLIDE 8

One Identity - Restricted

8

Log aggregation

■ Elastic stack on top of a local syslog: ■ Also most LaaS adds an additional layer on top of existing log management

slide-9
SLIDE 9

One Identity - Restricted

9

Why is it a problem?

■ More computing resources ■ More network bandwidth (cloud!) ■ More human resources ■ More security problems

slide-10
SLIDE 10

One Identity - Restricted

10

Using a unifjed log management layer

■ Saves on computing, network & human resources ■ Easier to push through security & operation teams ■ Log management is separate from analytics ■ Bonus: might save on analytics licensing and hardware costs

slide-11
SLIDE 11

#GetIAMRight | One Identity - Restricted - Confjdential

11

Implementing log management on syslog-ng

■ What is syslog-ng ■ Four roles: collecting, processing, fjltering, store/forward ■ Modes of operation ■ Confjguration

slide-12
SLIDE 12

One Identity - Restricted

12

syslog-ng

Logging

Recording events, such as:

Jan 14 11:38:48 linux-0jbu sshd[7716]: Accepted publickey for root from 127.0.0.1 port 48806 ssh2

syslog-ng

Enhanced logging daemon with a focus on portability and high-performance central log collection. Originally developed in C.

slide-13
SLIDE 13

One Identity - Restricted

13

Role: data collector

Collect system and application logs together: contextual data for either side A wide variety of platform-specifjc sources: ■/dev/log & co ■Journal, Sun streams Receive syslog messages over the network: ■Legacy or RFC5424, UDP/TCP/TLS Logs or any kind of text data from applications: ■Through fjles, sockets, pipes, application output, etc. Python source: Jolly Joker ■HTTP server, Kafka source, etc.

slide-14
SLIDE 14

One Identity - Restricted

14

Role: processing

Classify, normalize, and structure logs with built-in parsers: ■ CSV-parser, PatternDB, JSON parser, key=value parser Rewrite messages: ■ For example: anonymization Reformatting messages using templates: ■ Destination might need a specifjc format (ISO date, JSON, etc.) Enrich data: ■ GeoIP ■ Additional fjelds based on message content Python parser: ■ all of above, enrich logs from databases and also fjltering

slide-15
SLIDE 15

One Identity - Restricted

15

Role: data fjltering

Main uses: ■ Discarding surplus logs (not storing debug-level messages) ■ Message routing (login events to SIEM) Many possibilities: ■ Based on message content, parameters, or macros ■ Using comparisons, wildcards, regular expressions, and functions ■ Combining all of these with Boolean operators

slide-16
SLIDE 16

One Identity - Restricted

16

Role: destinations

slide-17
SLIDE 17

MODES OF OPERATION

  • Client mode: collecting logs from the client and sending them to

the remote server (directly or through a relay)

  • Relay mode: collecting logs from the clients (through the network)

and sending them to the remote server (directly or through another relay)

  • Server mode: collecting logs from the clients and storing them

locally or in a database

slide-18
SLIDE 18

One Identity - Restricted

18

Why relays?

UDP source Collect as close

as possible

Scalability

Distributing processing

Structure

A relay for each site or department

slide-19
SLIDE 19

One Identity - Restricted

19

Freeform log messages

Most log messages are: date + hostname + text Mar 11 13:37:56 linux-6965 sshd[4547]: Accepted keyboard- interactive/pam for root from 127.0.0.1 port 46048 ssh2 ■ Text = English sentence with some variable parts ■ Easy to read by a human ■ Diffjcult to create alerts or reports

slide-20
SLIDE 20

One Identity - Restricted

20

Solution: structured logging

■ Events represented as name-value pairs. For example, an ssh login: ■ app=sshd user=root source_ip=192.168.123.45 ■ syslog-ng: name-value pairs inside ■ Date, facility, priority, program name, pid, etc. ■ Parsers in syslog-ng can turn unstructured and some structured data (CSV, JSON) into name-value pairs ■ ■ Name-value pairs make fjltering more precise

slide-21
SLIDE 21

#GetIAMRight | One Identity - Restricted - Confjdential

21

Confjguration

■ “Don't Panic” ■ Simple and logical, even if it looks diffjcult at fjrst ■ Pipeline model:

■

Many different building blocks (sources, destinations, fjlters, parsers, etc.)

■

Connected into a pipeline using “log” statements

slide-22
SLIDE 22

One Identity - Restricted

22

syslog-ng.conf: getting started

@version:3.18 @include "scl.conf" # this is a comment :)

  • ptions {fmush_lines (0); keep_hostname (yes);};

source s_sys { system(); internal();}; destination d_mesg { fjle("/var/log/messages"); }; fjlter f_default { level(info..emerg) and not (facility(mail)); }; log { source(s_sys); fjlter(f_default); destination(d_mesg); }; @include "/etc/syslog-ng/conf.d/*.conf"

slide-23
SLIDE 23

One Identity - Restricted

23

Suricata.conf: source, JSON parsing

# receive Suricata logs source s_suricata { tcp(ip("0.0.0.0") port("514") fmags(no-parse)); }; # parse JSON into name-value pairs parser p_json { json-parser (prefjx("suricata.")); };

slide-24
SLIDE 24

One Identity - Restricted

24

Suricata.conf: GeoIP

parser p_geoip2 { geoip2( "${suricata.dest_ip}", prefjx( "parsed.dest." ) database( "/usr/share/GeoIP/GeoLite2-City.mmdb" ) ); }; rewrite r_geoip2 { set( "${parsed.dest.location.latitude},${parsed.dest.location.longitude}", value( "parsed.dest.ll" ), condition(not "${parsed.dest.location.latitude}" == "") ); };

slide-25
SLIDE 25

One Identity - Restricted

25

Suricata.conf: destinations

destination d_suricata { fjle("/var/log/suricata.log" template("$(format-json --key suricata.* --key parsed.* --key ISODATE)\n")); }; destination d_elastic { elasticsearch2 ( cluster("syslog-ng") client_mode("http") index("syslog") time-zone(UTC) type("syslog") fmush-limit(1) server("192.168.1.187") template("$(format-json --key suricata.* --key parsed.* --key ISODATE)") persist-name(elasticsearch-syslog) ) };

slide-26
SLIDE 26

One Identity - Restricted

26

Suricata.conf: more parsers

# resolve non-local destination IP addresses using Python parser parser p_resolver { python(class("SngResolver")); }; # add-contextual-data based on local IP address parser p_localsrc_info { add-contextual-data(selector("${suricata.src_ip}"), default- selector("unknown"), database("/etc/syslog-ng/conf.d/context-info-db.csv"), prefjx("parsed.src.")); };

slide-27
SLIDE 27

One Identity - Restricted

27

Suricata.conf: inline Python code

python { import socket class SngResolver(object): def parse(self, log_message): ipaddr_b = log_message['suricata.dest_ip'] ipaddr = ipaddr_b.decode('utf-8') try: resolved = socket.gethostbyaddr(ipaddr) hostname = resolved[0] log_message['parsed.dest.hostname'] = hostname except: pass return True };

slide-28
SLIDE 28

One Identity - Restricted

28

Suricata.conf: log statement 1.

log { # receive Suricata logs source(s_suricata); # parse JSON into name-value pairs parser(p_json); # resolve non-local destination IP addresses # using Python parser if (not match("^192.168" value("suricata.dest_ip"))) { parser(p_resolver); };

slide-29
SLIDE 29

One Identity - Restricted

29

Suricata.conf: log statement 2.

# add-contextual-data based on local IP address if (match("^192.168" value("suricata.src_ip"))) { parser(p_localsrc_info); }; # send alert if someone is reading slashdot if (match("slashdot.org" value("suricata.tls.sni"))) { destination { fjle("/var/log/slashdot"); }; # ToDo: change to smtp destination };

slide-30
SLIDE 30

One Identity - Restricted

30

Suricata.conf: log statement 3.

# talking to a malware C&C if { fjlter { in-list("/etc/syslog-ng/conf.d/malwarecc.list", value("suricata.dest_ip")) }; rewrite { set("Problem", value("parsed.malware")); }; } else { rewrite { set("OK", value("parsed.malware")); }; }; # add GeoIP information parser(p_geoip2); rewrite(r_geoip2);

slide-31
SLIDE 31

One Identity - Restricted

31

Suricata.conf: log statement 4.

# save results locally destination(d_suricata); # save results to Elasticsearch destination(d_elastic); };

slide-32
SLIDE 32

One Identity - Restricted

32

slide-33
SLIDE 33

#GetIAMRight | One Identity - Restricted - Confjdential

33

syslog-ng benefjts

High-performance reliable log collection Simplifjed architecture

Single application for both syslog and application data

Easier-to-use data

Parsed and presented in a ready-to-use format

Lower load on destinations

Effjcient message fjltering and routing

slide-34
SLIDE 34

#GetIAMRight | One Identity - Restricted - Confjdential

34

Join the community!

■ syslog-ng: http://syslog-ng.com/ ■ Source on GitHub: https://github.com/syslog-ng/syslog-ng ■ Mailing list: https://lists.balabit.hu/pipermail/syslog-ng/ ■ Gitter: https://gitter.im/syslog-ng/syslog-ng

slide-35
SLIDE 35

Questions?

syslog-ng blog: https://syslog-ng.com/community/ My e-mail: peter.czanik@oneidentity.com Twitter: https://twitter.com/PCzanik