Reconciling Performance and Security in High Load Environments - - PowerPoint PPT Presentation

▶
reconciling performance and security in high load
SMART_READER_LITE
LIVE PREVIEW

Reconciling Performance and Security in High Load Environments - - PowerPoint PPT Presentation

Mar 10, 2023 101 likes •1.45k views

Reconciling Performance and Security in High Load Environments Ignat Korchagin @ignatkn $ whoami Performance and security at Cloudflare Passionate about security and crypto Enjoy low level programming @ignatkn Performance vs

slide-1
SLIDE 1

Reconciling Performance and Security in High Load Environments

Ignat Korchagin @ignatkn

slide-2
SLIDE 2

@ignatkn

$ whoami

  • Performance and security at Cloudflare
  • Passionate about security and crypto
  • Enjoy low level programming
slide-3
SLIDE 3

Performance vs Security

slide-4
SLIDE 4

@ignatkn

Performance vs Security

slide-5
SLIDE 5

@ignatkn

security performance

Performance vs Security

slide-6
SLIDE 6

@ignatkn

Performance AND Security

slide-7
SLIDE 7

@ignatkn

Performance definition

  • performance in the narrow sense

○ speed ○ throughput ○ latency

slide-8
SLIDE 8

@ignatkn

Performance definition

  • performance in the narrow sense

○ speed ○ throughput ○ latency

  • performance in the broader sense

○ all above ○ resource optimisation ○ process optimisation ○ etc

slide-9
SLIDE 9

0-cost security

slide-10
SLIDE 10

@ignatkn

0-cost security

  • security cost is negligible and/or affects some

non-primary metric

slide-11
SLIDE 11

@ignatkn

0-cost security

  • security cost is negligible and/or affects some

non-primary metric

  • security cost is hidden/amortised by the architecture
  • r implementation
slide-12
SLIDE 12

@ignatkn

0-cost security

  • security cost is negligible and/or affects some

non-primary metric

  • security cost is hidden/amortised by the architecture
  • r implementation
  • the cost is not incurred for normal system behaviour

(prohibitive security)

slide-13
SLIDE 13

@ignatkn

Negligible security cost: secure boot chain

system firmware

slide-14
SLIDE 14

@ignatkn

Negligible security cost: secure boot chain

system firmware bootloader verify

slide-15
SLIDE 15

@ignatkn

Negligible security cost: secure boot chain

system firmware bootloader

  • perating

system verify verify

slide-16
SLIDE 16

@ignatkn

Negligible security cost: secure boot chain

system firmware bootloader

  • perating

system drivers verify verify verify

slide-17
SLIDE 17

@ignatkn

Negligible security cost: secure boot chain

system firmware bootloader

  • perating

system drivers applications, services verify verify verify verify

slide-18
SLIDE 18

@ignatkn

Negligible security cost: secure boot chain

system firmware bootloader

  • perating

system drivers applications, services verify verify verify verify

slide-19
SLIDE 19

@ignatkn

Negligible security cost: secure boot chain

system firmware bootloader

  • perating

system drivers applications, services verify verify verify verify

slide-20
SLIDE 20

@ignatkn

Negligible security cost: secure boot chain

  • ensures all running code is authorised by the system
  • wner
slide-21
SLIDE 21

@ignatkn

Negligible security cost: secure boot chain

  • ensures all running code is authorised by the system
  • wner
  • most effective protection from persistent malware
slide-22
SLIDE 22

@ignatkn

Negligible security cost: secure boot chain

  • ensures all running code is authorised by the system
  • wner
  • most effective protection from persistent malware
  • enforces operational procedures

○ all changes are properly fixed in the VCS ○ no possibility for one-off fixes ○ systems run only what’s needed

slide-23
SLIDE 23

@ignatkn

Negligible security cost: secure boot chain

  • ensures all running code is authorised by the system
  • wner
  • most effective protection from persistent malware
  • enforces operational procedures

○ all changes are properly fixed in the VCS ○ no possibility for one-off fixes ○ systems run only what’s needed

  • affects system boot time only

○ adds at most ~ms boot time

slide-24
SLIDE 24

@ignatkn

Amortised security cost: data encryption at rest

applications

slide-25
SLIDE 25

@ignatkn

Amortised security cost: data encryption at rest

filesystems applications

slide-26
SLIDE 26

@ignatkn

Amortised security cost: data encryption at rest

block subsystem filesystems applications

slide-27
SLIDE 27

@ignatkn

Amortised security cost: data encryption at rest

storage hardware block subsystem filesystems applications

slide-28
SLIDE 28

@ignatkn

Amortised security cost: data encryption at rest

storage hardware block subsystem filesystems applications

SED, OPAL

slide-29
SLIDE 29

@ignatkn

Amortised security cost: data encryption at rest

storage hardware block subsystem filesystems applications

SED, OPAL LUKS/dm-crypt, BitLocker, FileVault

slide-30
SLIDE 30

@ignatkn

Amortised security cost: data encryption at rest

storage hardware block subsystem filesystems applications

SED, OPAL LUKS/dm-crypt, BitLocker, FileVault ecryptfs, ext4 encryption

  • r fscrypt
slide-31
SLIDE 31

@ignatkn

Amortised security cost: data encryption at rest

storage hardware block subsystem filesystems applications

SED, OPAL LUKS/dm-crypt, BitLocker, FileVault ecryptfs, ext4 encryption

  • r fscrypt

DBMS, PGP, OpenSSL, Themis

slide-32
SLIDE 32

@ignatkn

Amortised security cost: data encryption at rest

storage hardware block subsystem filesystems applications

SED, OPAL LUKS/dm-crypt, BitLocker, FileVault ecryptfs, ext4 encryption

  • r fscrypt

DBMS, PGP, OpenSSL, Themis

slide-33
SLIDE 33

@ignatkn

Amortised security cost: data encryption at rest

Advantages of OS full disk encryption

slide-34
SLIDE 34

@ignatkn

Amortised security cost: data encryption at rest

Advantages of OS full disk encryption

  • little configuration needed
slide-35
SLIDE 35

@ignatkn

Amortised security cost: data encryption at rest

Advantages of OS full disk encryption

  • little configuration needed
  • fully transparent to applications
slide-36
SLIDE 36

@ignatkn

Amortised security cost: data encryption at rest

Advantages of OS full disk encryption

  • little configuration needed
  • fully transparent to applications
  • don’t roll our own crypto (unlike application layer)
slide-37
SLIDE 37

@ignatkn

Amortised security cost: data encryption at rest

Advantages of OS full disk encryption

  • little configuration needed
  • fully transparent to applications
  • don’t roll our own crypto (unlike application layer)
  • open, audible (unlike hardware layer)
slide-38
SLIDE 38

@ignatkn

What is a CDN?

https://en.wikipedia.org/wiki/Content_delivery_network

slide-39
SLIDE 39

@ignatkn

What is a CDN?

https://en.wikipedia.org/wiki/Content_delivery_network

slide-40
SLIDE 40

@ignatkn

Cloudflare Network

slide-41
SLIDE 41

@ignatkn

Average CDN cache response tail latency

  • encrypted (vanilla LUKS/dm-crypt)
  • unencrypted
slide-42
SLIDE 42

@ignatkn

Average CDN cache response tail latency

  • encrypted (vanilla LUKS/dm-crypt)
  • unencrypted
  • encrypted (patched LUKS/dm-crypt)

https://www.usenix.org/conference/vault20/presentation/korchagin

slide-43
SLIDE 43

@ignatkn

Disk encryption overhead

  • expected lower disk encryption overhead

○ got none ○ no changes in crypto algorithms, formats etc

slide-44
SLIDE 44

@ignatkn

Disk encryption overhead

  • expected lower disk encryption overhead

○ got none ○ no changes in crypto algorithms, formats etc

  • zero overhead data encryption is a no-brainer
slide-45
SLIDE 45

@ignatkn

Disk encryption overhead

  • expected lower disk encryption overhead

○ got none ○ no changes in crypto algorithms, formats etc

  • zero overhead data encryption is a no-brainer
  • encourages further performance improvement

research

○ data encryption is not the bottleneck anymore ○ indicates potential room for performance improvements for the

  • verall system
slide-46
SLIDE 46

@ignatkn

Prohibitive security: syscalls

application

slide-47
SLIDE 47

@ignatkn

OS kernel

Prohibitive security: syscalls

application

slide-48
SLIDE 48

@ignatkn

OS kernel

Prohibitive security: syscalls

application

  • pen

read write send recv accept

slide-49
SLIDE 49

@ignatkn

OS kernel

Prohibitive security: syscalls

application

  • pen

read write send recv accept

slide-50
SLIDE 50

@ignatkn

OS kernel

Prohibitive security: syscalls and seccomp

application

  • pen

read write send recv accept

Contract

  • open
  • read
  • write
slide-51
SLIDE 51

@ignatkn

OS kernel

Prohibitive security: syscalls and seccomp

application

  • pen

read write send recv accept

Contract

  • open
  • read
  • write
slide-52
SLIDE 52

@ignatkn

OS kernel

Prohibitive security: syscalls and seccomp

application

  • pen

read write send recv accept

Contract

  • open
  • read
  • write
slide-53
SLIDE 53

@ignatkn

Prohibitive security: syscalls and seccomp

slide-54
SLIDE 54

@ignatkn

Prohibitive security: syscalls and seccomp

slide-55
SLIDE 55

@ignatkn

Prohibitive security: syscalls and seccomp

Hi! I’m a clock app. I will only use gettimeofday

slide-56
SLIDE 56

@ignatkn

Prohibitive security: syscalls and seccomp

Hi! I’m a clock app. I will only use gettimeofday

gettimeofday

slide-57
SLIDE 57

@ignatkn

Prohibitive security: syscalls and seccomp

Hi! I’m a clock app. I will only use gettimeofday

gettimeofday 1970-01-01T00:00:00Z

slide-58
SLIDE 58

@ignatkn

Prohibitive security: syscalls and seccomp

Hi! I’m a clock app. I will only use gettimeofday

gettimeofday 1970-01-01T00:00:00Z

slide-59
SLIDE 59

@ignatkn

Prohibitive security: syscalls and seccomp

Hi! I’m a clock app. I will only use gettimeofday

gettimeofday 1970-01-01T00:00:00Z send

slide-60
SLIDE 60

@ignatkn

Prohibitive security: syscalls and seccomp

Hi! I’m a clock app. I will only use gettimeofday

gettimeofday 1970-01-01T00:00:00Z send

slide-61
SLIDE 61

@ignatkn

Prohibitive security: syscalls and seccomp

  • greatly limits the potential damage of RCE exploits
slide-62
SLIDE 62

@ignatkn

Prohibitive security: syscalls and seccomp

  • greatly limits the potential damage of RCE exploits
  • 0-cost overhead

○ no security is triggered for expected system behaviour

slide-63
SLIDE 63

@ignatkn

Prohibitive security: syscalls and seccomp

  • greatly limits the potential damage of RCE exploits
  • 0-cost overhead

○ no security is triggered for expected system behaviour

  • improves development velocity

○ developer intent vs actual implementation

slide-64
SLIDE 64

Security and systems performance

slide-65
SLIDE 65

@ignatkn

HTTP/2 and HTTP/3

HTTP/2 (2015)

slide-66
SLIDE 66

@ignatkn

HTTP/2 and HTTP/3

HTTP/2 (2015)

  • major rework from HTTP/1 (1991)

○ binary protocol ○ connection multiplexing ○ server push

slide-67
SLIDE 67

@ignatkn

HTTP/2 and HTTP/3

HTTP/2 (2015)

  • major rework from HTTP/1 (1991)

○ binary protocol ○ connection multiplexing ○ server push

  • HTTP/3 (in progress)

○ transport over QUIC/UDP

slide-68
SLIDE 68

@ignatkn

HTTP/2 performance (2015)

https://blog.cloudflare.com/introducing-http2/

slide-69
SLIDE 69

@ignatkn

HTTP/2 performance

https://imagekit.io/demo/http2-vs-http1

slide-70
SLIDE 70

@ignatkn

HTTP/2 performance

https://www.flickr.com/photos/smemon/15944989872/

slide-71
SLIDE 71

@ignatkn

HTTP/2 performance

https://www.flickr.com/photos/smemon/15944989872/

slide-72
SLIDE 72

@ignatkn

SSL/TLS

https://www.cloudflare.com/learning/ssl/what-happens-in-a-tls-handshake/

slide-73
SLIDE 73

@ignatkn

SSL/TLS: RSA vs ECC

https://www.cloudflare.com/learning/ssl/what-happens-in-a-tls-handshake/

slide-74
SLIDE 74

@ignatkn

SSL/TLS: RSA vs ECC

  • RSA

○ “older” cryptosystem (1977) ○ factoring problem of large numbers ○ sub-exponential complexity cracking algorithms ○ large keys (>=2048 bit)

slide-75
SLIDE 75

@ignatkn

SSL/TLS: RSA vs ECC

  • RSA

○ “older” cryptosystem (1977) ○ factoring problem of large numbers ○ sub-exponential complexity cracking algorithms ○ large keys (>=2048 bit)

  • ECC

○ “newer” cryptosystem (1985) ○ discrete logarithm problem over elliptic curves ○ exponential complexity cracking algorithms ○ small keys (>=256 bit)

slide-76
SLIDE 76

@ignatkn

SSL/TLS: RSA vs ECC

$ openssl speed rsa ecdsa

slide-77
SLIDE 77

@ignatkn

SSL/TLS: RSA vs ECC

$ openssl speed rsa ecdsa sign verify sign/s verify/s ... rsa 2048 bits 0.000616s 0.000018s 1623.5 55200.6 ... 256 bit ecdsa (nistp256) 0.0000s 0.0001s 25487.6 10731.6

slide-78
SLIDE 78

@ignatkn

SSL/TLS: RSA vs ECC

  • faster TLS handshakes (~15 times faster from above)

$ openssl speed rsa ecdsa sign verify sign/s verify/s ... rsa 2048 bits 0.000616s 0.000018s 1623.5 55200.6 ... 256 bit ecdsa (nistp256) 0.0000s 0.0001s 25487.6 10731.6

slide-79
SLIDE 79

@ignatkn

SSL/TLS: RSA vs ECC

  • faster TLS handshakes (~15 times faster from above)
  • less CPU utilisation

$ openssl speed rsa ecdsa sign verify sign/s verify/s ... rsa 2048 bits 0.000616s 0.000018s 1623.5 55200.6 ... 256 bit ecdsa (nistp256) 0.0000s 0.0001s 25487.6 10731.6

slide-80
SLIDE 80

@ignatkn

SSL/TLS: RSA vs ECC

  • faster TLS handshakes (~15 times faster from above)
  • less CPU utilisation
  • less key storage

$ openssl speed rsa ecdsa sign verify sign/s verify/s ... rsa 2048 bits 0.000616s 0.000018s 1623.5 55200.6 ... 256 bit ecdsa (nistp256) 0.0000s 0.0001s 25487.6 10731.6

slide-81
SLIDE 81

@ignatkn

SSL/TLS: RSA vs ECC

  • faster TLS handshakes (~15 times faster from above)
  • less CPU utilisation
  • less key storage
  • better security

$ openssl speed rsa ecdsa sign verify sign/s verify/s ... rsa 2048 bits 0.000616s 0.000018s 1623.5 55200.6 ... 256 bit ecdsa (nistp256) 0.0000s 0.0001s 25487.6 10731.6

slide-82
SLIDE 82

@ignatkn

SSL/TLS: RSA vs ECC (2017)

https://blog.cloudflare.com/how-expensive-is-crypto-anyway/

slide-83
SLIDE 83

@ignatkn

The Internet: network of networks

https://www.cloudflare.com/en-au/learning/security/glossary/what-is-bgp/

slide-84
SLIDE 84

@ignatkn

The Internet: AS and BGP

I have 1.1.1.1

slide-85
SLIDE 85

@ignatkn

The Internet: AS and BGP

I have 1.1.1.1 I have 8.8.8.8

slide-86
SLIDE 86

@ignatkn

The Internet: packet switching

By Oddbodz - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=29033823

slide-87
SLIDE 87

@ignatkn

The Internet: BGP security

I have 1.1.1.1 I have 8.8.8.8

slide-88
SLIDE 88

@ignatkn

The Internet: BGP security

I have 1.1.1.1 I have 8.8.8.8 I have 1.2.3.4

slide-89
SLIDE 89

@ignatkn

The Internet: BGP security

I have 1.1.1.1 I have 8.8.8.8 I have 1.2.3.4 I have 1.2.3.4

slide-90
SLIDE 90

@ignatkn

The Internet: BGP with RPKI

I have 1.1.1.1 I have 8.8.8.8 I have 1.2.3.4 I have 1.2.3.4

slide-91
SLIDE 91

@ignatkn

The Internet: BGP with RPKI

I have 1.1.1.1 I have 8.8.8.8 I have 1.2.3.4 I have 1.2.3.4

slide-92
SLIDE 92

@ignatkn

The Internet: BGP with RPKI

  • RPKI prevents bad actors from claiming resources

they don’t own

slide-93
SLIDE 93

@ignatkn

The Internet: BGP with RPKI

  • RPKI prevents bad actors from claiming resources

they don’t own

  • however, not all “false claimers” are bad actors

○ bugs in network equipment software ○ network equipment misconfigurations

slide-94
SLIDE 94

@ignatkn

The Internet: BGP with RPKI

  • RPKI prevents bad actors from claiming resources

they don’t own

  • however, not all “false claimers” are bad actors

○ bugs in network equipment software ○ network equipment misconfigurations

  • RPKI improves network throughput by ensuring

routes validity

○ some misconfigurations cause severe outages ○ minor misconfigurations create packet loss

https://blog.cloudflare.com/how-verizon-and-a-bgp-optimizer-knocked-large-parts-of-the-internet-offline-today/

slide-95
SLIDE 95

Security and process performance

slide-96
SLIDE 96

@ignatkn

Cloudflare Network

slide-97
SLIDE 97

@ignatkn

Datacentre provisioning

  • connect hardware
slide-98
SLIDE 98

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware
slide-99
SLIDE 99

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network

slide-100
SLIDE 100

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network ○ configure OOB

slide-101
SLIDE 101

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network ○ configure OOB ○ secure OOB

slide-102
SLIDE 102

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network ○ configure OOB ○ secure OOB ○ dump serial numbers

slide-103
SLIDE 103

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network ○ configure OOB ○ secure OOB ○ dump serial numbers ○ cross-check with the inventory system

slide-104
SLIDE 104

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network ○ configure OOB ○ secure OOB ○ dump serial numbers ○ cross-check with the inventory system

  • initial key provisioning

○ ssh and/or configuration management ○ verify and authorise key fingerprints

slide-105
SLIDE 105

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network ○ configure OOB ○ secure OOB ○ dump serial numbers ○ cross-check with the inventory system

  • initial key provisioning

○ ssh and/or configuration management ○ verify and authorise key fingerprints

slide-106
SLIDE 106

@ignatkn

Datacentre provisioning

  • connect hardware
  • verify hardware

○ setup initial network ○ configure OOB ○ secure OOB ○ dump serial numbers ○ cross-check with the inventory system

  • initial key provisioning

○ ssh and/or configuration management ○ verify and authorise key fingerprints

slide-107
SLIDE 107

@ignatkn

What is a TPM?

slide-108
SLIDE 108

@ignatkn

What is a TPM?

  • tamper resistant crypto chip in modern laptops and

servers

slide-109
SLIDE 109

@ignatkn

What is a TPM?

  • tamper resistant crypto chip in modern laptops and

servers

  • can provide secure key storage and hardware

random number generator

slide-110
SLIDE 110

@ignatkn

What is a TPM?

  • tamper resistant crypto chip in modern laptops and

servers

  • can provide secure key storage and hardware

random number generator

  • fundamental building block for remote attestation

○ authenticated identity for remote systems ○ trustworthy assertions about the state of the remote systems

slide-111
SLIDE 111

@ignatkn

Remote attestation

verifier

slide-112
SLIDE 112

@ignatkn

Remote attestation

verifier remote host

TPM

slide-113
SLIDE 113

@ignatkn

Remote attestation

verifier remote host

TPM

quote

slide-114
SLIDE 114

@ignatkn

Remote attestation

verifier remote host

TPM

quote

slide-115
SLIDE 115

@ignatkn

Remote attestation

verifier remote host

TPM

quote

slide-116
SLIDE 116

@ignatkn

Remote attestation

verifier remote host

TPM

quote

slide-117
SLIDE 117

@ignatkn

Remote attestation

verifier remote host

TPM

quote

  • we’re communicating with the right host
slide-118
SLIDE 118

@ignatkn

Remote attestation

verifier remote host

TPM

quote

  • we’re communicating with the right host
  • we’re communicating with the right host securely
slide-119
SLIDE 119

@ignatkn

Remote attestation

verifier remote host

TPM

quote

  • we’re communicating with the right host
  • we’re communicating with the right host securely
  • the remote host runs only authorised software

○ firmware ○

  • perating system

○

  • ther software
slide-120
SLIDE 120

@ignatkn

Datacentre provisioning with TPM

slide-121
SLIDE 121

@ignatkn

Datacentre provisioning with TPM

  • verify server identity
slide-122
SLIDE 122

@ignatkn

Datacentre provisioning with TPM

  • verify server identity
  • verify running OS
slide-123
SLIDE 123

@ignatkn

Datacentre provisioning with TPM

  • verify server identity
  • verify running OS
  • cross-check serial numbers
slide-124
SLIDE 124

@ignatkn

Datacentre provisioning with TPM

  • verify server identity
  • verify running OS
  • cross-check serial numbers
  • provision configuration

management keys

slide-125
SLIDE 125

@ignatkn

Datacentre provisioning with TPM

  • verify server identity
  • verify running OS
  • cross-check serial numbers
  • provision configuration

management keys

  • start serving production traffic
slide-126
SLIDE 126

@ignatkn

Datacentre provisioning with TPM

  • verify server identity
  • verify running OS
  • cross-check serial numbers
  • provision configuration

management keys

  • start serving production traffic
slide-127
SLIDE 127

@ignatkn

Datacentre provisioning with TPM

  • verify server identity
  • verify running OS
  • cross-check serial numbers
  • provision configuration

management keys

  • start serving production traffic
slide-128
SLIDE 128

@ignatkn

Datacentre provisioning with TPM

  • better automation

○ less room for human errors or misconfigurations

slide-129
SLIDE 129

@ignatkn

Datacentre provisioning with TPM

  • better automation

○ less room for human errors or misconfigurations

  • faster datacentre provisioning

○ from weeks to days

slide-130
SLIDE 130

@ignatkn

Datacentre provisioning with TPM

  • better automation

○ less room for human errors or misconfigurations

  • faster datacentre provisioning

○ from weeks to days

  • efficient engineering time

○ engineers can develop/improve systems rather than do repetitive tasks

slide-131
SLIDE 131

@ignatkn

Datacentre provisioning with TPM

  • better automation

○ less room for human errors or misconfigurations

  • faster datacentre provisioning

○ from weeks to days

  • efficient engineering time

○ engineers can develop/improve systems rather than do repetitive tasks

  • better security
slide-132
SLIDE 132

@ignatkn

Conclusions

  • security does not always have to impact performance

○ 0-cost security

  • sometimes security actually improves performance
  • security can improve performance in the broader

sense and in longer term

  • “performance by security” approach is useful in

driving and prioritising company security improvements

slide-133
SLIDE 133

Questions?