Achieving Keyless CDNs with Conclaves Stephen Herwig Christina - - PowerPoint PPT Presentation

Achieving Keyless CDNs with Conclaves

Stephen Herwig Dave Levin Christina Garman

User Bank

customer’s

• rigin server

Content Delivery Networks host their customers’ websites

CDNs

customer’s

• rigin server

CDN’s edge server

Content Delivery Networks host their customers’ websites

CDNs CDNs reduce page load times

CDN’s edge server

customer’s

• rigin server

CDNs CDNs reduce page load times

CDN’s edge server

customer’s

• rigin server

CDNs CDNs mitigate and block attacks

CDN’s edge server

customer’s

• rigin server

CDNs CDNs mitigate and block attacks

CDN’s edge server

customer’s

• rigin server

Customers share their keys with CDNs

CDN’s edge server

bank’s private key

Customers share their keys with CDNs

CDN’s edge server

Key sharing is widespread

Cangialosi et al., CCS 2016

Key sharing is widespread

0.2 0.4 0.6 0.8 1 200k 400k 600k 800k 1M Fraction of Domains Hosted

• n Third-party Providers

Alexa Site Rank (bins of 10,000) At least one key shared All keys shared

43% of the top 10k
 most popular websites

Cangialosi et al., CCS 2016

Key sharing is widespread

0.2 0.4 0.6 0.8 1 200k 400k 600k 800k 1M Fraction of Domains Hosted

• n Third-party Providers

Alexa Site Rank (bins of 10,000) At least one key shared All keys shared

43% of the top 10k
 most popular websites

Cangialosi et al., CCS 2016

The web has consolidated keys in the hands of a few CDNs

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin)

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) Key server performs actions requiring private key

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) Key server performs actions requiring private key

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) Key server performs actions requiring private key

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) Key server performs actions requiring private key

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) Key server performs actions requiring private key

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) Key server performs actions requiring private key

Key server performs actions requiring private key

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) The CDN learns all session keys

Key server performs actions requiring private key

Keyless SSL

Introduced by Cloudflare to mitigate key sharing

Private keys stay at the key server (origin) The CDN learns all session keys

In practice:
 CDN

Third-party resources

• n

using Legacy applications Can we Maintain privacy ?

Third-party resources Legacy applications Maintain privacy

The CDN is no more trusted
 than a standard on-path attacker

Third-party resources Legacy applications Maintain privacy

The CDN is no more trusted
 than a standard on-path attacker No changes to existing code-bases;
 facilitates deployment and adoption

Third-party resources Legacy applications Maintain privacy

The CDN is no more trusted
 than a standard on-path attacker Leverage the existing infrastructure. One additional assumption: TEEs No changes to existing code-bases;
 facilitates deployment and adoption

Third-party resources Legacy applications Maintain privacy

The CDN is no more trusted
 than a standard on-path attacker Leverage the existing infrastructure. One additional assumption: TEEs No changes to existing code-bases;
 facilitates deployment and adoption

Third-party resources Legacy applications Maintain privacy

The CDN is no more trusted
 than a standard on-path attacker Leverage the existing infrastructure. One additional assumption: TEEs No changes to existing code-bases;
 facilitates deployment and adoption Phoenix

Trusted execution environments

Hardware Operating
 System Application

By default, assume all system components are untrusted

Code Service

Trusted execution environments

Hardware Operating
 System Application

By default, assume all system components are untrusted

Small trusted CPU
 Resistant to physical attacks

Code Service

Trusted execution environments

Hardware Operating
 System Application

By default, assume all system components are untrusted

Small trusted CPU
 Resistant to physical attacks

Code Service

Enclave: Isolated
 application memory

Enclave

Trusted execution environments

Hardware Operating
 System Application

By default, assume all system components are untrusted

Small trusted CPU
 Resistant to physical attacks

Code Service

Model: Code and data can safely reside inside an enclave

Enclave: Isolated
 application memory

Enclave

Hardware Operating
 System Application

Code Service

Practical limitations of TEEs

Syscalls Untrusted

Applications inside enclaves cannot make syscalls

Enclave

Hardware Operating
 System

Service

libOSes

Idea: Implement a small “OS” inside the enclave

Enclave

Hardware Operating
 System

Service

Application

Code

libOS

Service

libOSes

Idea: Implement a small “OS” inside the enclave

Enclave

Hardware Operating
 System

Service

Application

Code

libOS

Service

libOSes

"Syscalls"

Idea: Implement a small “OS” inside the enclave

Enclave

Hardware Operating
 System

Service

Application

Code

libOS

Service

libOSes

"Syscalls"

Idea: Implement a small “OS” inside the enclave

Enclave

Service locally
 when possible

Hardware Operating
 System

Service

Application

Code

libOS

Service

libOSes

"Syscalls"

Idea: Implement a small “OS” inside the enclave

Enclave

Service locally
 when possible Syscalls

Graphene-SGX

Tsai et al., ATC 2017

A libOS for Intel SGX that supports some services

Graphene-SGX

A libOS for Intel SGX that supports some services

fork exec

Graphene’s supported services:

pipes, signals, semaphores

Tsai et al., ATC 2017

Graphene-SGX

A libOS for Intel SGX that supports some services

fork exec

Graphene’s supported services:

pipes, signals, semaphores

What constitutes a CDN?

Cache Web Application
 Firewall Key Server Web server

Multiple
 tenants Needs
 plaintext Needs
 disk Needs 
 safe
 storage

Graphene-SGX

A libOS for Intel SGX that supports some services

fork exec

Graphene’s supported services:

pipes, signals, semaphores

What constitutes a CDN?

Cache Web Application
 Firewall Key Server Reading & writing files Shared memory

Also critical to a CDN:

Access to private keys Web server

Multiple
 tenants Needs
 plaintext Needs
 disk Needs 
 safe
 storage

The first truly keyless CDN

fork exec

Graphene’s supported services:

pipes, signals, semaphores

What constitutes a CDN?

Cache Web Application
 Firewall Key Server Web server

Multiple
 tenants Needs
 plaintext Needs
 disk Needs 
 safe
 storage

Phoenix

Containers of enclaves Conclaves

The first truly keyless CDN

fork exec

Graphene’s supported services:

pipes, signals, semaphores

What constitutes a CDN?

Cache Web Application
 Firewall Key Server Reading & writing files Shared memory

Also critical to a CDN:

Access to private keys Web server

Multiple
 tenants Needs
 plaintext Needs
 disk Needs 
 safe
 storage

Phoenix

Containers of enclaves Conclaves

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

TLS

Enclaves mutually
 authenticate via attested TLS

Knauth et al., 2018

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

TLS

Private key operation

Enclaves mutually
 authenticate via attested TLS

Knauth et al., 2018

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

TLS

Private key operation Result

Enclaves mutually
 authenticate via attested TLS

Knauth et al., 2018

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Shared memory read

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Shared memory read Shared memory write

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Application

Code

libOS

Service Enclave Enclave

Memory Server

Hardware Operating
 System

Service

Shared memory

Conclaves

Application libOS

Service Enclave Enclave

Memory Server

Hardware Operating
 System

Service fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

Hardware Operating
 System

Service fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

RPC

Hardware Operating
 System

Service fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

RPC

Hardware Operating
 System

Service

Coordinates locks
 Maintains memory locations

fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

Hardware Operating
 System

Service

Coordinates locks
 Maintains memory locations

fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

Hardware Operating
 System

Service

Syscall

Coordinates locks
 Maintains memory locations

fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

Hardware Operating
 System

Service

Syscall

Coordinates locks
 Maintains memory locations

fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

Hardware Operating
 System

Service

Syscall

Memory file
 Encrypted on untrusted disk Coordinates locks
 Maintains memory locations

fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave

"Syscall"

Enclave

Memory Server

Hardware Operating
 System

Service

Syscall

Memory file
 Encrypted on untrusted disk Coordinates locks
 Maintains memory locations

fcntl()

Shared memory

Conclaves

Application libOS

Service Enclave Enclave

Memory Server

Hardware Operating
 System

Service

Memory file
 Encrypted on untrusted disk Coordinates locks
 Maintains memory locations

Shared
 Memory fcntl()

Shared memory

Conclaves

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Enclave

File Server

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Enclave

File Server

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Enclave

File Server

File access

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Insight: Treat enclaves like a distributed system
 Implement services using kernel servers

Enclave

Memory Server

Enclave

File Server

File access Verified data

Application

Code

libOS

Service Enclave Enclave

File Server

Hardware Operating
 System

Service

File system access

Conclaves

Application

Code

libOS

Service Enclave Enclave

File Server

File system access

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

Application libOS

Service Enclave Enclave

File Server

Hardware Operating
 System

Service read()

Conclaves

Merkle Tree
 Encrypted on untrusted disk

File system access

Application libOS

Service Enclave

"Syscall"

Enclave

File Server

Hardware Operating
 System

Service read()

Conclaves

Merkle Tree
 Encrypted on untrusted disk

File system access

Application libOS

Service Enclave

"Syscall"

Enclave

File Server

RPC

Hardware Operating
 System

Service read()

Conclaves

Merkle Tree
 Encrypted on untrusted disk

File system access

Application libOS

Service Enclave

"Syscall"

Enclave read()

libOS

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

RPC

ext2fs server Block layer

Merkle root

File system access

Application libOS

Service Enclave

"Syscall"

Enclave read()

libOS

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

RPC

Syscall

ext2fs server Block layer

Merkle root

File system access

Application libOS

Service Enclave

"Syscall"

Enclave read()

libOS

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

RPC

Syscall

ext2fs server Block layer

Merkle root

File system access

Application libOS

Service Enclave

"Syscall"

Enclave read()

libOS

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

RPC

Syscall

ext2fs server Block layer

Merkle root

File system access

Application libOS

Service Enclave

"Syscall"

Enclave read()

libOS

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

RPC

Syscall

Verifies branches Decrypts blocks

ext2fs server Block layer

Merkle root

File system access

Application libOS

Service Enclave

"Syscall"

Enclave read()

libOS

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

RPC

Syscall

Verifies branches Decrypts blocks

ext2fs server Block layer

Merkle root

File system access

Application libOS

Service Enclave read()

File system access

Conclaves

Hardware Operating
 System

Service

Merkle Tree
 Encrypted on untrusted disk

Enclave

ext2fs server libOS Block layer

Merkle root

Data

Verifies branches Decrypts blocks

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall Key Server

Enclave

Web server

Cache

Web Application
 Firewall

Execution environment is a distributed system of enclaves

Enclave

Memory Server

Enclave

File Server

The first truly keyless CDN

Phoenix

Containers of enclaves Conclaves

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall Key Server

Enclave

Web server

Cache

Web Application
 Firewall

Execution environment is a distributed system of enclaves

Enclave

Memory Server

Enclave

File Server

Conclave

The first truly keyless CDN

Phoenix

Containers of enclaves Conclaves

Enclave Enclave

Web server

Cache

Web Application
 Firewall

Enclave

Web server

Cache

Web Application
 Firewall

The first truly keyless CDN

Key Server

Phoenix

Containers of enclaves Conclaves

Enclave

Web server

Cache

Web Application
 Firewall

Execution environment is a distributed system of enclaves

Enclave

Memory Server

Enclave

File Server

Conclave

fork exec

Conclaves supported services:

pipes, signals, semaphores Reading & writing files Shared memory Access to private keys Trusted time server

The first truly keyless CDN

Phoenix

The first truly keyless CDN

Phoenix

Websites delegate provisioning to CDNs Phoenix supports many deployment configurations

Other details in the paper

Supports multi-tenancy Both CDN and website can store private data

The first truly keyless CDN

Phoenix

Websites delegate provisioning to CDNs Phoenix supports many deployment configurations

Other details in the paper

Conclave Conclave Conclave

Supports multi-tenancy Both CDN and website can store private data

The first truly keyless CDN

Phoenix

Conclave Conclave Conclave

ARTIFACT EVALUATED

PASSED

Implemented on top of Graphene-SGX Evaluated to understand throughput and scalability

500 1000 1500 2000 2500 3000 3500 1 KiB 10 KiB 100 KiB

# Workers

1 2 4 8

Throughput (requests/sec) Downloaded file size

Fetch a file 10,000 times over non-persistent HTTPS connections from among 128 concurrent clients

What is Phoenix’s request throughput?

500 1000 1500 2000 2500 3000 3500 1 KiB 10 KiB 100 KiB

# Workers

1 2 4 8

Throughput (requests/sec) Downloaded file size

Linux

Fetch a file 10,000 times over non-persistent HTTPS connections from among 128 concurrent clients

NGINX running on normal Linux

What is Phoenix’s request throughput?

500 1000 1500 2000 2500 3000 3500 1 KiB 10 KiB 100 KiB

# Workers

1 2 4 8

Throughput (requests/sec) Downloaded file size

Linux Phoenix-crypt Phoenix-vericrypt

Fetch a file 10,000 times over non-persistent HTTPS connections from among 128 concurrent clients

confidentiality confidentiality & integrity

What is Phoenix’s request throughput?

How does Phoenix scale to multiple tenants?

200 400 600 800 1000 1200 1400 1600 1 2 4 6 Time per request (ms) Number of tenants Linux (shared NGINX)

40 ms 264 ms

200 400 600 800 1000 1200 1400 1600 1 2 4 6 Time per request (ms) Number of tenants Linux (shared NGINX) Phoenix-crypt (shared nothing)

8 16 32 48

How does Phoenix scale to multiple tenants?

264 ms 40 ms 127 ms 1437 ms

Number of enclaves

200 400 600 800 1000 1200 1400 1600 1 2 4 6 Time per request (ms) Number of tenants Linux (shared NGINX) Phoenix-crypt (shared NGINX)

8 10 14 18

Phoenix-crypt (shared nothing)

8 16 32 48

Number of enclaves

How does Phoenix scale to multiple tenants?

264 ms 40 ms 128 ms 127 ms 1437 ms 806 ms

200 400 600 800 1000 1200 1400 1600 1 2 4 6 1 1K 10K 100K 1M 10M Time per request (ms) SGX paging events Number of tenants Linux (shared NGINX) Phoenix-crypt (shared NGINX)

8 10 14 18

Phoenix-crypt (shared nothing)

8 16 32 48

How does Phoenix scale to multiple tenants?

Number of enclaves

264 ms 40 ms 128 ms 806 ms 127 ms 1437 ms

Other results

Benchmark overhead of running WAFs (ModSecurity) in SGX (overhead about the same as in Linux) Perform detailed micro-benchmarks of each kernel server Compare standard ocalls to exitless ocalls (not always better)

ARTIFACT EVALUATED

PASSED

Moderate performance overheads

https://phoenix.cs.umd.edu/

500 1000 1500 2000 2500 3000 3500 1 KiB 10 KiB 100 KiB

# Workers

1 2 4 8

Throughput (requests/sec) Downloaded file size

Linux Phoenix-crypt Phoenix-vericrypt

The first truly keyless CDN

Phoenix

Containers of enclaves Conclaves

Run legacy apps in enclaves