DIY Patch Management F L O R I A N J U N G E ( @ S H A N T YC O D - - PowerPoint PPT Presentation

diy patch management
SMART_READER_LITE
LIVE PREVIEW

DIY Patch Management F L O R I A N J U N G E ( @ S H A N T YC O D - - PowerPoint PPT Presentation

DIY Patch Management F L O R I A N J U N G E ( @ S H A N T YC O D E ) I N G O B E N T E ( @ I N G O B E N T E ) B S I D E S M U N I C H 2 0 1 8 Patching Isnt that solved? Nope, its not. R E M E M B E R T H O S E R A N S O M


slide-1
SLIDE 1

DIY Patch Management

F L O R I A N J U N G E ( @ S H A N T YC O D E ) I N G O B E N T E ( @ I N G O B E N T E ) B S I D E S M U N I C H 2 0 1 8

slide-2
SLIDE 2

Patching


Isn’t that solved?

slide-3
SLIDE 3

Nope, it’s not.

R E M E M B E R T H O S E R A N S O M WA R E N E W S I N 2 0 1 7 ?

slide-4
SLIDE 4

SINNERSCHRADER

BSides Munich | DIY Patch Management | 2018 4

https://en.wikipedia.org/wiki/WannaCry_ransomware_attack

slide-5
SLIDE 5

SINNERSCHRADER

Example WannaCry

  • WannaCry hit the world on May 12 2017
  • It spread via a vulnerability called EternalBlue
  • EternalBlue was fixed by Microsoft on March 14 2017
slide-6
SLIDE 6

SINNERSCHRADER

Example WannaCry

  • WannaCry hit the world on May 12 2017
  • It spread via a vulnerability called EternalBlue
  • EternalBlue was fixed by Microsoft on March 14 2017
slide-7
SLIDE 7

2 months between patch and outbreak

I . E . YO U CA N WO R RY L E S S A B O U T 0 DAYS : )

slide-8
SLIDE 8

Patching is hard

AT L E AST I N M A N Y R E A L WO R L D S C E N A R I O S.

slide-9
SLIDE 9

But why?

I M E A N I T WO R KS O N O U R FA M I LY M E M B E R S C O M P U T E R S, TO O.

slide-10
SLIDE 10

Constraints

L E GAC Y : E N D O F L I F E O S T H AT I S N OT PATC H A B L E .

slide-11
SLIDE 11

Constraints

AVA I L A B I L I T Y : H OW TO R E B O OT T H AT H Y P E RV I S O R C L U ST E R ?

slide-12
SLIDE 12

Constraints

M O N E Y : I T WO R KS W I T H O U T T H E PATC H , D O E S N ’ T I T ?

slide-13
SLIDE 13

So what now?

L E T ’ S TA K E A G L I M P S E I N TO O U R WO R L D.

slide-14
SLIDE 14

SinnerSchrader Ecosystem

slide-15
SLIDE 15

SINNERSCHRADER

In a world where …

  • N tenants
  • M tech stacks
  • N x M requirements
slide-16
SLIDE 16

SINNERSCHRADER

In a world where …

  • Heterogenous infrastructure
  • OS-wise mostly Debian and Ubuntu
  • Packed into VMs and Containers
  • Yes, there is also some serverless stuff :)
slide-17
SLIDE 17

In a world where …

Inconsistent patch management

N E V E R , S O M E T I M E S, R E G U L A R LY.

slide-18
SLIDE 18

In a world where …

Commercial scanners?

N O B U D G E T. W E A R E N OT T H AT E N T E R P R I SY.

slide-19
SLIDE 19

SINNERSCHRADER

Lessons learned … so far

  • Installation is easy
  • Patching is hard
  • Knowing when to patch is even harder
slide-20
SLIDE 20

Not cool

S O W E WA N T E D TO C H A N G E T H AT.

slide-21
SLIDE 21

SINNERSCHRADER

Solution - the easy part

  • Manually scan for all the CVEs
  • Automate CVE scans (i.e. daily)
  • Gather all the logs
slide-22
SLIDE 22

SINNERSCHRADER

Solution - the tricky part

  • Dashboard everything
  • Get metrics that CXOs can understand
  • Take action (i.e. patch) and check the metrics
  • Lean back … for now
slide-23
SLIDE 23

Building blocks

slide-24
SLIDE 24

SINNERSCHRADER

  • CVE scanner to audit VMs
  • Integration to config management
  • Central logging and dashboarding
slide-25
SLIDE 25

SINNERSCHRADER

Spot the vuln - the audit

  • Vulnerability databases
  • Vulnerability scanner
  • Vulnerability subscriptions
  • Freemium pricing model
  • Nice people :)
slide-26
SLIDE 26

SINNERSCHRADER

Spot the vuln - the audit

  • nmap plugin
  • Burp plugin
  • getsploit
  • API
slide-27
SLIDE 27

SINNERSCHRADER

Spot the vuln - CVE scanner

  • Get installed packages
  • Audit each for CVEs
  • Get CVSS scores
slide-28
SLIDE 28

Demo

V U L N E R S. C O M A P I

slide-29
SLIDE 29

SINNERSCHRADER

Orchestration via config management

  • Do it!
  • Our solution: SaltStack
  • Codify your update strategy

https://docs.saltstack.com/en/getstarted/overview.html

slide-30
SLIDE 30

SINNERSCHRADER

Orchestration

  • Define systems with formula
  • Minion matching
  • Template engine

https://docs.saltstack.com/en/getstarted/overview.html

slide-31
SLIDE 31

SINNERSCHRADER

Three patch management flavours

  • Unattended upgrades
  • Orchestrated updates
  • Patch Day

https://en.wikipedia.org/wiki/Neapolitan_ice_cream

slide-32
SLIDE 32

SINNERSCHRADER

Centralized Logging

  • Elasticsearch
  • Logstash
  • Kibana

figure based on https://www.elastic.co/guide/en/logstash/current/introduction.html

slide-33
SLIDE 33

SINNERSCHRADER

slide-34
SLIDE 34

Demo

K I B A N A . S Z O P S. D E

slide-35
SLIDE 35 BSides Munich | DIY Patch Management | 2018

SINNERSCHRADER

The big picture

35
slide-36
SLIDE 36 BSides Munich | DIY Patch Management | 2018

SINNERSCHRADER

The big picture

36
slide-37
SLIDE 37 BSides Munich | DIY Patch Management | 2018

SINNERSCHRADER

The big picture

37
slide-38
SLIDE 38

Limitations

slide-39
SLIDE 39

There are some limitations

A K A ST U F F T H AT C O M P L I A N C E FO L KS L I K E LY WO N ’ T F I N D, B U T T H AT YO U K N OW I S T H E R E …

slide-40
SLIDE 40

SINNERSCHRADER

Tales from the kernel

  • Notice version encoded in package name
  • This confuses vulners (no CVEs)
  • As well as unattended upgrades (yep, no upgrades)
slide-41
SLIDE 41

SINNERSCHRADER

Tales from the kernel - the fix

  • Install the meta package linux-image-generic
slide-42
SLIDE 42 BSides Munich | DIY Patch Management | 2018

SINNERSCHRADER

Reboot hassle

42

  • Running old kernel although newer one is installed
slide-43
SLIDE 43 BSides Munich | DIY Patch Management | 2018

SINNERSCHRADER

Reboot hassle - the fix

43

  • Monitor uptime of your servers ;-)
  • The two metrics that matter for host security by

Diogo Monica

slide-44
SLIDE 44

Next steps

slide-45
SLIDE 45

SINNERSCHRADER

Next steps

  • Fix some quirks ;-)
  • Container checks with Claire
  • AWS integration
  • nsp check
slide-46
SLIDE 46

SINNERSCHRADER

Alternatives

  • OpenVAS / vuls
  • https://github.com/0x4D31/salt-scanner
  • Your-typical-Enterprise-Distribution-Mgmt-here
  • Reverse uptime & Golden image freshness
slide-47
SLIDE 47

SINNERSCHRADER

Kudos

  • Kirill Ermakov from Vulners.com(@isox_xx)
  • Christoph Trautwein (@trautw) & the S2 ops crew
slide-48
SLIDE 48

Thanks!