Base64 is not encryption A better story for Kubernetes secrets - - PowerPoint PPT Presentation

▶
base64 is not encryption
SMART_READER_LITE
LIVE PREVIEW

Base64 is not encryption A better story for Kubernetes secrets - - PowerPoint PPT Presentation

Jan 14, 2023 145 likes •598 views

Base64 is not encryption A better story for Kubernetes secrets @sethvargo Developer Relations Engineer What's a secret? Secret (noun) Credentials, configurations, API keys, or other pieces of information needed by an application at build

slide-1
SLIDE 1

Base64 is not encryption

A better story for Kubernetes secrets

slide-2
SLIDE 2

@sethvargo

Developer Relations Engineer

slide-3
SLIDE 3

What's a secret?

slide-4
SLIDE 4

Secret (noun)

Credentials, configurations, API keys, or other pieces of information needed by an application at build time or run time

slide-5
SLIDE 5

Why protect secrets?

  • Attractive target for hackers
  • Often leaked in repos or storage buckets
  • Frequently includes overly broad permissions
slide-6
SLIDE 6

Protecting secrets

Audit

Verify and log the use

  • f individual secrets to

a central system

Encrypt

Always encrypt secrets in transit with TLS and at rest

Rotate

Change a secret regularly or in case of suspected compromise

Isolate

Separate where secrets are used from where secrets are managed

slide-7
SLIDE 7

Protecting secrets

Audit

Verify and log the use

  • f individual secrets to

a central system

Encrypt

Always encrypt secrets in transit with TLS and at rest

Rotate

Change a secret regularly or in case of suspected compromise

Isolate

Separate where secrets are used from where secrets are managed

slide-8
SLIDE 8

Layers of encryption

Application-layer encryption Service-level encryption Filesystem encryption Machine-level encryption

slide-9
SLIDE 9

App-layer encryption

  • Applied at earliest possible step
  • Provides protection a very granular level
  • Protects data as it moves through the system
slide-10
SLIDE 10

Kubernetes defaults

slide-11
SLIDE 11

Insecure by default

Secrets are stored in plaintext in etcd. They are base64-encoded, but not encrypted.

slide-12
SLIDE 12

Insecure by default*

Secrets are stored in plaintext in etcd. They are base64-encoded, but not encrypted. * Many providers alter this default behavior.

slide-13
SLIDE 13

==

kube-apiserver etcd Master

slide-14
SLIDE 14

kube-apiserver etcd Master

==

slide-15
SLIDE 15

Encraption

shodan.io/search?query=etcd

slide-16
SLIDE 16

Demo

slide-17
SLIDE 17

Envelope encryption

slide-18
SLIDE 18

DEK KEK

Key encryption key Data encryption key

Envelope encryption

Data

slide-19
SLIDE 19

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100001 01110100 01100001 01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000

Encrypted data Encrypted DEK

01100101 01101110 01100101 01101110 01100011 01110010 01100011 01110010 01111001 01110000 01111001 01110000 01110100 01100101 01110100 01100101 01100100 00100000 01100100 00100000 01100100 01100001 01100100 01100101 01110100 01100001 01101011 00100000

Storage

slide-20
SLIDE 20

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100001 01110100 01100001 01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000

Encrypted data Encrypted DEK

slide-21
SLIDE 21
slide-22
SLIDE 22

Envelope encryption

  • Generate unique DEKs for each data entry
  • Crypto-shred - revoke KEK and data is gone
  • Easy versioning and rotation
slide-23
SLIDE 23

Kubernetes 1.7

Envelope encryption

slide-24
SLIDE 24

kind: EncryptionConfiguration apiVersion: apiserver.config.k8s.io/v1 resources:

  • resources:
  • secrets

providers:

  • aescbc:

keys:

  • name: key1

secret: 9RlIhvmh1e6+Ixv0CjyUkA==

  • name: key2

secret: u+aswHTypAyoRKH5/P0r5A==

  • secretbox:

keys:

  • name: key1

secret: 9aHuiH/wrlmWEXZp9br4og==

slide-25
SLIDE 25

./kube-apiserver \

  • -encryption-provider-config=/etc/encryption-config.yaml \
  • -other-options...
slide-26
SLIDE 26

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000 EncryptionConfiguration

kube-apiserver etcd Master

slide-27
SLIDE 27

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000 EncryptionConfiguration

kube-apiserver etcd Master

slide-28
SLIDE 28

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000 EncryptionConfiguration

kube-apiserver etcd Master

slide-29
SLIDE 29

Drawbacks

  • Need to generate keys yourself
  • Key management is your responsibility
  • Rotation is a manual process (and tedious)
  • No HSM integration
slide-30
SLIDE 30

Drawbacks

The underlying encryption keys are still stored in plaintext on the filesystem!

slide-31
SLIDE 31

Kubernetes 1.10

KMS encryption providers

slide-32
SLIDE 32

kind: EncryptionConfiguration apiVersion: apiserver.config.k8s.io/v1 resources:

  • resources:
  • secrets

providers:

  • kms:

name: myKmsPlugin endpoint: unix:///tmp/kms-socketfile.sock cachesize: 100

slide-33
SLIDE 33

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000 EncryptionConfiguration

kube-apiserver etcd Master KMS

slide-34
SLIDE 34

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000 EncryptionConfiguration

kube-apiserver etcd Master KMS

slide-35
SLIDE 35

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000 EncryptionConfiguration

kube-apiserver etcd Master KMS

slide-36
SLIDE 36

Existing plugins (GitHub)

  • GoogleCloudPlatform/k8s-cloudkms-plugin
  • Azure/kubernetes-kms
  • kubernetes-sigs/aws-encryption-provider
  • oracle/kubernetes-vault-kms-plugin
slide-37
SLIDE 37

GKE Integration (beta)

gcloud beta container clusters create my-cluster

  • -database-encryption-key-location us-east1
  • -database-encryption-key-keyring my-keyring
  • -database-encryption-key my-crypto-key
slide-38
SLIDE 38

Initial secret problem?

  • IAM can solve the "first secret" problem
  • Delegate PAM to the cloud provider via IAM
  • Separate concerns: etcd nodes don't need IAM

permissions to talk to KMS

slide-39
SLIDE 39

Vault

slide-40
SLIDE 40

01100101 01101110 01100011 01110010 01111001 01110000 01110100 01100101 01100100 00100000 01100100 01100101 01101011 00100000 EncryptionConfiguration

kube-apiserver etcd Master KMS

slide-41
SLIDE 41

Demo

slide-42
SLIDE 42

Summary

slide-43
SLIDE 43

Summary

  • Use at least two layers of encryption
  • Rotate keys regularly
  • Leverage envelope encryption
  • Protect K8S secrets using an external KMS
slide-44
SLIDE 44

@sethvargo

Developer Relations Engineer

Thanks!