[PPT] - BabelFish: Fusing Address Translations for Containers Dimitrios PowerPoint Presentation

SLIDE 1

BabelFish: Fusing Address Translations for Containers

Dimitrios Skarlatos, Umur Darbaz, Bhargava Gopireddy, Nam Sung Kim, and Josep Torrellas University of Illinois at Urbana-Champaign skarlat2@illinois.edu ISCA 2020

SLIDE 2

Conventional Cloud Computing

2

Hardware

Hypervisor

Lib App

Guest OS

Lib App

Guest OS

Virtual Machines

SLIDE 3

New Era in Cloud Computing

3

Lib App

Container Engine

Lib App Lib App Hardware

Operating System

Containers

1. Lightweight
2. Faster bringup
3. Higher consolidation

OS automatically shares resources!

SLIDE 4

Containers in the Cloud

4

Replicated Containers Scalability Fault Tolerance Load Balancing

Pod

High number of containers Very frequent context switches Serverless & Function-as-a-Service (FaaS)

Cold start effects
Very short runtime

Shared Data Pages

Library

Container Engine

Hardware

Operating System

Application Binary

SLIDE 5

Problem: Replicated VAàPA Translations

5

Replicated Containers Scalability Fault Tolerance Load Balancing

Pod

High number of containers Very frequent context switches Serverless & Function-as-a-Service (FaaS)

Cold start effects
Very short runtime

Shared Data Pages

Library

Container Engine

Hardware

Operating System

Application Binary

1. TLB thrashing
2. Redundant page table work

SLIDE 6

Solution: Share VAàPA Translations

6

Replicated Containers Scalability Fault Tolerance Load Balancing

Pod

High number of containers Very frequent context switches Serverless & Function-as-a-Service (FaaS)

Cold start effects
Very short runtime

Shared Data Pages

Library

Container Engine

Hardware

Operating System

Application Binary

1. Minimize TLB thrashing
2. Eliminate redundant page table work

SLIDE 7

Contribution: BabelFish

HW and OS support to share translations across containers

1. Introduce Container Context IDentifiers (CCID)
2. Extend TLB design
3. Share page tables

Performance improvement

Data-serving: 11%-18% Compute: 11% Function-as-a-Service: 10-55% Container bring-up: 8%

7

“The Babel fish is small, yellow, leech-like, and probably the oddest thing in the Universe. The practical upshot of all this is that if you stick a Babel fish in your ear you can instantly understand anything said to you in any form of language.” The Hitchhiker's Guide to the Galaxy – Douglas Adams

SLIDE 8

Problem: TLB Bloat

8

Main Memory Core L3 Cache L1 Cache L2 Cache TLB

Container A

Page Tables A

PA 4 VA 1

PA 4

SLIDE 9

Problem: TLB Bloat

9

Main Memory

PA 4

Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

à “Page Walk” = Fetch entry from page table TLB Miss Issue LD VA 1

PA4 VA1

Container A

SLIDE 10

Problem: TLB Bloat

10

Main Memory

PA 4

Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Container A

PA4 VA1 PCID

Existing Process Context IDentifiers (PCID)

SLIDE 11

Problem: TLB Bloat

11

Main Memory

PA 4

Page Tables B Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Container B

PA4 VA1 PCID

What if we could identify shared entries?

PA4 VA1

Issue LD VA 1 à “Page Walk” = Fetch entry from page table TLB Miss

PCID PA4 VA1

SLIDE 12

Problem: TLB Bloat

12

Main Memory

PA 4

Page Tables B Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Container B

PA4 VA1 PA4 VA1

What if we could identify shared entries?

SLIDE 13

Solution: BabelFish TLB

13

Main Memory

PA 4

Page Tables B Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Container B

PA4 VA1 PA4 VA1

Unlimited sharing of translations among any number of containers! Container A Container Context Identifier (CCID)

CCID

SLIDE 14

Solution: BabelFish TLB

14

Main Memory

PA 4

Page Tables B Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Container B

PA4 VA1 PA4 VA1

Container A Container Context Identifier (CCID) VAàPA shared for some containers & private for others Container C PrivateCopy (PC)

CCID PC

SLIDE 15

Solution: BabelFish TLB

15

Main Memory

PA 4

Page Tables B Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Container B

PA4 VA1 PA4 VA1

Container A Container Context Identifier (CCID) PrivateCopy (PC) VAàPA shared for some containers & private for others

PA6 VA1 CCID PC

Container C

CCID PC

SLIDE 16

Problem: Page Table Entry Bloat

16

Main Memory

PA 4

Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

PA4 VA1 PCID

Container A

PA 4 VA 1 PA 4 VA 1

PA4 VA1

SLIDE 17

Problem: Page Table Entry Bloat

17

Main Memory

PA 4

Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

PA4 VA1 PCID PA4 VA1 PCID

Page Tables B

PA 4 VA 1 PA 4 VA 1 PA 4 VA 1 PA 4 VA 1 PA 4 VA 1

PA4 VA1

Container B

SLIDE 18

Problem: Lazy Page Table Management

18

Main Memory

PA 4

Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Page Tables B

PA 4 VA 1 P

Container A

P

Present bit

SLIDE 19

Problem: Lazy Page Table Management

PA 4

19

Main Memory Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Page Tables B

PA 4 VA 1 P

Container A

P

Present bit currently cleared

SLIDE 20

Problem: Lazy Page Table Management

20

Main Memory Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Page Tables B

PA 4 VA 1 P

Container A

P

Issue LD VA 1

PA4 VA1

P

Page Fault!

Storage PA 4

SLIDE 21

Problem: Lazy Page Table Management

21

Main Memory

PA 4

Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Page Tables B

PA 4 VA 1 P

Container A

P

Issue LD VA 1 Page Fault!

Storage Present bit set Present bit remains cleared

P

SLIDE 22

Problem: Lazy Page Table Management

22

Main Memory

PA 4

Page Tables A Core L3 Cache L1 Cache L2 Cache TLB

PA 4 VA 1

Page Tables B

PA 4 VA 1 P P

Page Fault!

Container B

Issue LD VA 1

PA4 VA1

P

Present bit set

P P

SLIDE 23

Conventional Page Tables

23

pmd_t pte_t PGD PUD PMD PTE CR30

47 … 39 38 … 30 29 … 21 20 … 12 11 … 0 9-bits 9-bits 9-bits 9-bits 12-bits

+ + + Virtual Address To TLB +

Container A

SLIDE 24

Conventional Page Tables

24

pmd_t pte_t PGD PUD PMD CR30

47 … 39 38 … 30 29 … 21 20 … 12 11 … 0 9-bits 9-bits 9-bits 9-bits 12-bits

+ + + Virtual Address To TLB + PTE pmd_t PGD PUD PMD CR31 + + + pte_t PTE + To TLB

Container A Container B

SLIDE 25

Conventional Page Tables

25

pmd_t pte_t PGD PUD PMD CR30

47 … 39 38 … 30 29 … 21 20 … 12 11 … 0 9-bits 9-bits 9-bits 9-bits 12-bits

+ + + Virtual Address To TLB + PTE pmd_t PGD PUD PMD CR31 + + + pte_t PTE + To TLB

Container A Container B

SLIDE 26

Solution: BabelFish Page Table Sharing

26

pmd_t pte_t PGD PUD PMD PTE CR30

47 … 39 38 … 30 29 … 21 20 … 12 11 … 0 9-bits 9-bits 9-bits 9-bits 12-bits

+ + + + Virtual Address To TLB pmd_t PGD PUD PMD CR31 + + +

Container A Container B

SLIDE 27

Example

28

Container A: Runs on Core 0, requests VAàPA Container B: Runs on Core 1, requests VAàPA Container C: Runs on Core 0, requests VAàPA

SLIDE 28

Example

29

Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

Time

Conventional

L2 Cache Miss, L3 Cache Miss, Main Memory Hit

Page Walk Cache (PWC)

SLIDE 29

Example

30

Issue LD V1 Core 0 L1 TLB Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

Time

Conventional

Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault L2 TLB Miss PWC Miss

SLIDE 30

Example

31

Issue LD V1 Core 0 L1 TLB Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

Time

Conventional

Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault L2 TLB Miss PWC Miss

SLIDE 31

Example

32

Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

Time

Conventional

Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

BabelFish

Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

SLIDE 32

Example

33

Issue LD V1 Core 0 L1 TLB Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

Time

Conventional

Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

BabelFish

Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk

PUD Walk PMD Walk PTE Walk

L2 Cache Miss, L3 Cache Hit

L2 TLB Miss PWC Miss

SLIDE 33

Example

34

Issue LD V1 Core 0 L1 TLB Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

Time

Conventional

Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

BabelFish

Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk

PUD Walk PMD Walk PTE Walk

L2 Cache Miss, L3 Cache Hit

Issue LD V1 Core 0 L1 TLB Hit L2 TLB Miss PWC Miss

SLIDE 34

Example

35

Issue LD V1 Core 0 L1 TLB Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

Time

Conventional

Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault

BabelFish

Issue LD V1 Core 0 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk PUD Walk PMD Walk PTE Walk Page Fault Issue LD V1 Core 1 L1 TLB Miss L2 TLB Miss PWC Miss PGD Walk

PUD Walk PMD Walk PTE Walk

L2 Cache Miss, L3 Cache Hit

Issue LD V1 Core 0 L1 TLB Hit

Result: Faster Execution Time!

L2 TLB Miss PWC Miss

SLIDE 35

Workloads & Methodology

Workloads:

Data serving: ArangoDB, MongoDB, HTTPd Compute: GraphChi, FIO FaaS: Parse, Hash, Marshal (both dense and sparse)

Full system simulations with Simics + SST

36

2 containers per core 3 containers per core

SLIDE 36

Page Table Sharing for Two Containers

37

5-minute run on a real machine

SLIDE 37

Page Table Sharing for Two Containers

38

5-minute run on a real machine

SLIDE 38

Page Table Sharing for Two Containers

39

5-minute run on a real machine

SLIDE 39

Page Table Sharing for Two Containers

40

Active

5-minute run on a real machine

SLIDE 40

Page Table Sharing for Two Containers

41

After Merging Active

5-minute run on a real machine

SLIDE 41

Page Table Sharing for Two Containers

42

Page Sharing is Common à Major Reduction in Translations

5-minute run on a real machine

After Merging Active

SLIDE 42

Performance

43

Major Performance Gains à Both from TLB and page tables

55%

SLIDE 43

More in the Paper

Copy-on-Write pages Address Space Layout Randomization (ASLR) for security Other security considerations More evaluation

Area, energy, sharing, comparison vs larger TLB

44

SLIDE 44

Takeaway: BabelFish

Sharing Translations in TLB and Page Tables

Introduces Container Context ID (CCID) at OS and HW

TLB extensions & Page Table sharing support

Substantial speedup in application execution and container bring-up

Future work à CCID in OS and HW

Virtualized environments and nested page tables
Other resources

SLIDE 45

BabelFish: Fusing Address Translations for Containers

Dimitrios Skarlatos, Umur Darbaz, Bhargava Gopireddy, Nam Sung Kim, and Josep Torrellas University of Illinois at Urbana-Champaign skarlat2@illinois.edu ISCA 2020