Shield your cluster Security with Elasticsearch Alexander Reelsen - - PowerPoint PPT Presentation

▶
shield your cluster
SMART_READER_LITE
LIVE PREVIEW

Shield your cluster Security with Elasticsearch Alexander Reelsen - - PowerPoint PPT Presentation

Dec 25, 2023 965 likes •1.52k views

Shield your cluster Security with Elasticsearch Alexander Reelsen @spinscale alex@elastic.co Agenda Why? How? Q & A What? Next? Who? About 2012 Elasticsearch got founded Series A investment Trainings Supports subscriptions

slide-1
SLIDE 1

Shield your cluster

Security with Elasticsearch

Alexander Reelsen @spinscale alex@elastic.co

slide-2
SLIDE 2

Agenda

Why? How? Next? What? Who? Q & A

slide-3
SLIDE 3

About

2012

Elasticsearch got founded Series A investment Trainings Supports subscriptions

slide-4
SLIDE 4

About

2012

Series B investment Kibana Elasticsearch for Apache Hadoop Integration Logstash Elasticsearch Clients

2013

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

slide-5
SLIDE 5

About

2012

Series C investment Marvel released

2013 2014

slide-6
SLIDE 6

About

2012

Shield goes GA First user conference & rebrand Found acquired Packetbeat joins Watcher in beta

2013 2014 2015

slide-7
SLIDE 7

About

2012 2013 2014 2015

Joined in March 2013 Working on Elasticsearch & Shield Development, Trainings, Conferences, Support, Blog posts We're hiring...

slide-8
SLIDE 8

Why? How? Next? What? Who? Q & A

slide-9
SLIDE 9

Why?

Elasticsearch: No security OOTB No encrypted communication No Authorization No Authentication No Audit Logging

slide-10
SLIDE 10

ES nginx client

Filter by HTTP method, URI or IP User management via basic auth Use aliases & filters

nginx in front

slide-11
SLIDE 11

ES nginx client

How to solve multi index operations?

nginx in front

GET /logs-2015.10.10,evil,logs-2015.10.11 { "query" : { "match_all": {} } }

slide-12
SLIDE 12

ES nginx client

How to solve bulk/multi operations?

nginx in front

{ "index" : { "_index" : "test1", "_type" : "type1", "_id" : "1" } } { "field1" : "value1" } { "delete" : { "_index" : "test2", "_type" : "type1", "_id" : "2" } } { "create" : { "_index" : "test3", "_type" : "type1", "_id" : "3" } } { "field1" : "value3" } { "update" : {"_id" : "1", "_type" : "type1", "_index" : "test4"} } { "doc" : {"field2" : "value2"} }

slide-13
SLIDE 13

ES nginx client

Prevent unwanted accesses

nginx in front

HTTP/Transport

slide-14
SLIDE 14

ES nginx client

nginx in front

Firewall

slide-15
SLIDE 15

Data ACL client

Configuration scattered across systems

  • perational overhead

IP Filtering

slide-16
SLIDE 16

Data ACL client

  • perational overhead

IP Filtering Directory

Configuration scattered across systems

slide-17
SLIDE 17

Why? How? Next? What? Who? Q & A

slide-18
SLIDE 18

How?

Elasticsearch modular & pluggable Security as a plugin HTTP + Transport protocols Integration into the ELK stack!

slide-19
SLIDE 19

How?

Elasticsearch Elasticsearch auth_token

Authentication Authorization

slide-20
SLIDE 20

How?

Elasticsearch Elasticsearch auth_token 200 OK

Authentication Authorization

slide-21
SLIDE 21

How?

Elasticsearch Elasticsearch auth_token 401 Unauthorized

Authentication Authorization

slide-22
SLIDE 22

How?

Getting up and running is easy Install elasticsearch 1.6

bin/plugin install elasticsearch/license/latest bin/plugin install elasticsearch/shield/latest

slide-23
SLIDE 23

Why? How? Next? What? Who? Q & A

slide-24
SLIDE 24

What?

IP Filtering Encrypted communication Authentication Authorization Audit Trail

slide-25
SLIDE 25

Configurable in elasticsearch.yml Can be updated dynamically via cluster update settings API

IP Filtering

shield.transport.filter: allow: "192.168.0.1" deny: "192.168.0.0/24"

slide-26
SLIDE 26

keystore required different config for HTTP and transport protocol (+profiles)

Encrypted communication

shield.ssl.keystore.path: /path/to/keystore.jks shield.ssl.keystore.password: secret shield.transport.ssl: true shield.http.ssl: true

slide-27
SLIDE 27

Authentication

"Who are you?" Auth mechanisms are called realms Available: esusers, ldap, ad, pki Realms can be chained Support for caching & API for clearing

slide-28
SLIDE 28

Authentication

shield.authc: realms: esusers: type: esusers

  • rder: 0

ldap1: type: ldap

  • rder: 1

enabled: false url: 'url_to_ldap1' ... ad1: type: active_directory

  • rder: 3

url: 'url_to_ad'

slide-29
SLIDE 29

ESusers realm

Local files, can be changed via CLI Elasticsearch watches file changes & reloads config/shield/users config/shield/users_roles

slide-30
SLIDE 30

ESusers realm

bin/shield/esusers useradd alex bin/shield/esusers roles alex -a admin -r user bin/shield/esusers list bin/shield/esusers userdel alex

slide-31
SLIDE 31

Fallback to configurable user Disabled by default

Anonymous access

shield.authc: anonymous: username: anonymous_user roles: role1, role2

slide-32
SLIDE 32

Authorization

"Are you allowed to do that?" File: config/shield/roles.yml admin: cluster: all indices: '*': all

slide-33
SLIDE 33

Role Based Access Control

role

named set of permissions

permission

set of cluster wide privileges set of indices/aliases specific privileges

privilege

set of one or more action names /_search ⬌ indices:data/read/search

slide-34
SLIDE 34

Role Based Access Control

admin: cluster: all indices: '*': all

role permission

slide-35
SLIDE 35

Authorization

user: indices: '*': read

events_user: indices: 'events_*': read

slide-36
SLIDE 36

Authorization

get_user: indices: 'events_index': 'indices:data/read/get'

logfile_user_readonly: indices: "logstash-201?-*": read

slide-37
SLIDE 37

Audit Trail

Writes an own audit log file Implemented as logger Logs different types of event based

  • n log level

(ip filtering, tampered requests, access denied, auth failed)

shield.audit.enabled: true

slide-38
SLIDE 38

Integration

Transport Client Logstash Kibana 3/4 Watcher Marvel

slide-39
SLIDE 39

Transport Client

TransportClient client = new TransportClient(builder() .put("cluster.name", "myClusterName") .put("shield.user", "test_user:changeme") .put("shield.ssl.keystore.path", "/path/to/client.jks") .put("shield.ssl.keystore.password", "password") .put("shield.transport.ssl", "true")) .addTransportAddress(new InetSocketTransportAddress("localhost", 9300));

slide-40
SLIDE 40

Why? How? Next? What? Who? Q & A

slide-41
SLIDE 41

Who?

Use-case 1: Monitoring application No write access Cluster Health Nodes stats/info Indices Stats

slide-42
SLIDE 42

Use-case 2: Logstash

No read access (unless input is used) Indices: Indexing Cluster: Index templates

slide-43
SLIDE 43

Use-case 3: Marvel

marvel_user: cluster: cluster:monitor/nodes/info, cluster:admin/plugin/license/get indices: '.marvel-*': all marvel_agent: cluster: indices:admin/template/get, indices:admin/template/put indices: '.marvel-*': indices:data/write/bulk, create_index

slide-44
SLIDE 44

Use-case 4: Ecommerce

bulk: indices: 'products_*': write, manage, read updater: indices: 'products': index, delete, indices:admin/optimize webshop: indices: 'products': search, get

slide-45
SLIDE 45

Use-case 4: Ecommerce

monitoring: cluster: monitor indices: '*': monitor sales_rep : indices: 'sales_*' : all 'social_events' : data_access, monitor

slide-46
SLIDE 46

Why? How? Next? What? Who? Q & A

slide-47
SLIDE 47

Next?

Simplify SSL configuration API driven user/role management Open up realms API Field-level security Index Audit Trail into ES

slide-48
SLIDE 48

Why? How? Next? What? Who? Q & A

slide-49
SLIDE 49

Q & A

Thanks for listening!

Alexander Reelsen @spinscale alex@elastic.co

We're hiring https://www.elastic.co/about/careers We're helping https://www.elastic.co/subscriptions

slide-50
SLIDE 50

Resources

Shield documentation

https://www.elastic.co/guide/en/shield/current/index.html

Shield: Security in ELK

https://www.elastic.co/elasticon/2015/sf/security-in-elk

Shield and Beyond: Recommendations for a Secure ELK Environment

https://www.elastic.co/webinars/shield-and-beyond

slide-51
SLIDE 51

Resources

https://discuss.elastic.co/c/shield

slide-52
SLIDE 52

Resources

slide-53
SLIDE 53

Resources

slide-54
SLIDE 54

Q & A

Thanks for listening!

Alexander Reelsen @spinscale alex@elastic.co

We're hiring https://www.elastic.co/about/careers We're helping https://www.elastic.co/subscriptions