MINIMIZING THE WINDOW OF COMPROMISE PRACTICAL MTLS Ying Li @cyli
PROBLEM TYPICAL MICROSERVICE ARCHITECTURE VPC S1 DB S2 S1 S3
PROBLEM VLAN-TASTIC MICROSERVICE ARCHITECTURE S1 DB S2 S1 S3
PROBLEM CORRECT MICROSERVICE ARCHITECTURE S1 DB S2 S1 S3
PROBLEM APPLICATION TLS LIFECYCLE Bootstrap Revoke Renew
PROBLEM BOOTSTRAP • CSR ➡ CA • Configuration
PROBLEM RENEW • Schedule
PROBLEM RENEW • Schedule • CSR ➡ CA • Configuration
PROBLEM RENEW • Schedule • CSR ➡ CA • Configuration • Restart
PROBLEM REVOKE • CRL • OCSP [Stapling]
PRINCIPLE AUTOMATE, AUTOMATE, AUTOMATE • Promotes adoption of mTLS
PRINCIPLE AUTOMATE, AUTOMATE, AUTOMATE • Promotes adoption of mTLS • Single location for private key
PRINCIPLE AUTOMATE, AUTOMATE, AUTOMATE • Promotes adoption of mTLS • Single location for private key • Shorter certificate expiry
SWARMKIT OVERVIEW https://github.com/docker/swarmkit
SWARMKIT OVERVIEW CLUSTER Worker Worker Manager Manager Manager Worker Worker Worker Worker
SWARMKIT OVERVIEW Node Node CLUSTER Worker Worker Manager Node Node Manager Manager Worker Worker Node Node Worker Worker
SWARMKIT OVERVIEW CLUSTER Worker Worker Manager raft store Manager Manager Worker Worker Worker Worker
SWARMKIT OVERVIEW CLUSTER Node Node CA raft store CA CA Node Node Node Node
SWARMKIT’S IMPLEMENTATION
SWARMKIT’S IMPLEMENTATION BOOTSTRAP Token Random Version Secret SWMTKN-1-mx8suomaom825bet6-cm6zts22rl4hly2 Known Hash Prefix of Root CA
SWARMKIT’S IMPLEMENTATION BOOTSTRAP CA 1. Retrieve, validate Root CA certificate. 1 Node
SWARMKIT’S IMPLEMENTATION BOOTSTRAP CA 1. Retrieve, validate Root CA certificate. 2 2. CSR + secret token ➡ CA. (TLS) 1 Node
SWARMKIT’S IMPLEMENTATION BOOTSTRAP CA 1. Retrieve and validate Root CA Public key material. 2 2. CSR + secret token ➡ CA. (TLS) 1 3 3. Get certificate. (TLS) Node
SWARMKIT’S IMPLEMENTATION RENEW 50% 80% Valid Valid From Until
SWARMKIT’S IMPLEMENTATION RENEW CA 1. CSR + ➡ CA. (mTLS) 2.Get certificate. (mTLS) 1 2 Node
SWARMKIT’S IMPLEMENTATION RENEW 1. Trigger extra leader election 2. Workers all need to reconnect Restart to managers 3. Reschedule work
SWARMKIT’S IMPLEMENTATION RENEW
SWARMKIT’S IMPLEMENTATION RENEW
SWARMKIT’S IMPLEMENTATION RENEW Server Existing connections New connections
SWARMKIT’S IMPLEMENTATION RENEW Client Existing connections New connections
SWARMKIT’S IMPLEMENTATION REVOKE
SWARMKIT’S IMPLEMENTATION REVOKE REMOVE CRLS, OCSP [Stapling]
SWARMKIT’S IMPLEMENTATION REMOVE NODE BLACKLIST Node ID Certificate Expiry a8h1vsk3k9o5nwea858ty9kma 2017-08-26 01:02:52 UTC k80l2au3yq9f7x6r2oca13vwt 2017-07-15 11:35:23 UTC n970d5be9ccgnreg4iti4jho3 2017-08-01 22:59:05 UTC
SWARMKIT’S IMPLEMENTATION REMOVE Worker/Manager Manager Request Validate node ID against blacklist Authorize role Perform work Response Worker/Manager Manager
SWARMKIT’S IMPLEMENTATION REMOVE BLACKLIST VS WHITELIST
SWARMKIT’S IMPLEMENTATION REMOVE Manager delayed join Manager Manager
SWARMKIT’S IMPLEMENTATION REMOVE Manager Manager Manager
PROBLEM Rotate CA
PROBLEM CA ROTATION • (conf.) All nodes: trust old and new CA 1 • (wait.) Verify all nodes
PROBLEM CA ROTATION • (conf.) All nodes: trust old and new CA 1 • (wait.) Verify all nodes • (conf.) All nodes: renew certificates 2 • (wait.) Verify all nodes
PROBLEM CA ROTATION • (conf.) All nodes: trust old and new CA 1 • (wait.) Verify all nodes • (conf.) All nodes: renew certificates 2 • (wait.) Verify all nodes • (conf.) All nodes: trust new CA only 3 • (wait.) Verify all nodes
PRINCIPLE CROSS-SIGNED INTERMEDIATE Root A Root Root B B Key Info: A Key Info: B X Signed by: A Signed by: B Leaf cert: X Signed by: B Root: B
PRINCIPLE CROSS-SIGNED INTERMEDIATE Root A Root Root B A Key Info: A Key Info: B X Signed by: A Signed by: B Leaf cert: X Signed by: B Root: A
PRINCIPLE CROSS-SIGNED INTERMEDIATE Root A Root A Key Info: A Root A Intermediate Root B’ B DN: A Signed by: A Key Info: B Key Info: B DN: B DN: B Signed by: A Signed by: B
PRINCIPLE CROSS-SIGNED INTERMEDIATE Root A Leaf cert: X Signed by: B’ Root A Intermediate B’ Root: A X
PRINCIPLE CROSS-SIGNED INTERMEDIATE Root A Leaf cert: X Signed by: B Root A Root B Intermediate B’ Root: B X
SWARMKIT’S IMPLEMENTATION CA ROTATION • (conf.) All nodes: trust old and new CA • (wait.) Verify all nodes • Generate cross-signed intermediate
SWARMKIT’S IMPLEMENTATION CA ROTATION • Generate cross-signed intermediate • (conf.) All nodes: renew certificates 1 • (wait.) Verify all nodes
SWARMKIT’S IMPLEMENTATION CA ROTATION • Generate cross-signed intermediate • (conf.) All nodes: renew certificates 1 • (wait.) Verify all nodes • (conf.) All nodes: trust new CA 2 • (wait.) Verify all nodes • Throw away cross-signed intermediate
SWARMKIT’S IMPLEMENTATION CA ROTATION: BEFORE ROTATION Node Trust Cluster Root A Root A Root: Trust Root: Root A Node TLS Cluster Root A Certificate: Cert Issuer: Root A Z
SWARMKIT’S IMPLEMENTATION CA ROTATION: START ROTATION Node Trust Cluster Root A Root A Root: Trust Root: Root A Root A Node TLS Cluster Certificate: Cert Issuer: Root A Root A Intermediate Z B’
SWARMKIT’S IMPLEMENTATION CA ROTATION: NODE CERT RENEWAL Node Trust Cluster Root A Root A Root: Trust Root: Root Root A Node TLS Cluster A Certificate: Cert Issuer: Root A Intermediate Root A B Intermediate B’ X
SWARMKIT’S IMPLEMENTATION CA ROTATION: NODE CERT RENEWAL Node1 Node2 Node3 Node4 Node5 Root A Root A Root A Root A Root A Trust Root Root Root Root Root Root A A A TLS A A Ro A Ro A Ro A Certificate Intermediate Intermediate Intermediate B B B Z Z X X X
SWARMKIT’S IMPLEMENTATION CA ROTATION: ROTATE TRUST ROOT Node Trust Cluster Root B Root B Root: Trust Root: Root A Root A Node TLS Cluster Certificate: Cert Issuer: Root A Intermediate Root B B Root A Intermediate B’ X
SWARMKIT’S IMPLEMENTATION CA ROTATION: ROTATE TRUST ROOT Node1 Node2 Node3 Node4 Node5 Root B Root B Root A Root A Root B Trust Root Root Root Root Root Root A A A A A TLS Ro A Ro A Ro A Ro A Ro A Certificate Intermediate Intermediate Intermediate Intermediate Intermediate B B B B B X X X X X
SWARMKIT’S IMPLEMENTATION CA ROTATION: FINISH ROOT ROTATION Node Trust Cluster Root B Root B Root: Trust Root: Root A Node TLS Cluster Root B Certificate: Cert Issuer: Root A Intermediate Root B B X
DEMO
SUMMARY MINIMIZING THE WINDOW OF COMPROMISE
SUMMARY MINIMIZING THE WINDOW OF COMPROMISE • automatic bootstrap, renewal • short certificate expiry
SUMMARY MINIMIZING THE WINDOW OF COMPROMISE • automatic bootstrap, renewal • short certificate expiry • certificate revocation • CA rotation
SUMMARY MORE INFORMATION https://github.com/docker/swarmkit https://diogomonica.com/2017/01/11/hitless-tls-certificate-rotation-in-go/ https://github.com/cloudflare/cfssl (@cyli)
Recommend
More recommend
