Coordinated and Efficient Huge Page Management with Ingens Youngjin - - PowerPoint PPT Presentation

▶
coordinated and efficient huge page management with ingens
SMART_READER_LITE
LIVE PREVIEW

Coordinated and Efficient Huge Page Management with Ingens Youngjin - - PowerPoint PPT Presentation

Feb 11, 2023 270 likes •683 views

Coordinated and Efficient Huge Page Management with Ingens Youngjin Kwon , Hangchen Yu, Simon Peter, Christopher J. Rossbach, and Emmett Witchel 1 High address translation cost Modern applications: large memory footprint, low memory access

slide-1
SLIDE 1

Coordinated and Efficient Huge Page Management with Ingens

Youngjin Kwon, Hangchen Yu, Simon Peter, Christopher J. Rossbach, and Emmett Witchel

1

slide-2
SLIDE 2

High address translation cost

  • Modern applications: large memory footprint, low memory access locality
  • TLB coverage using base pages is insufficient

2

Cpu cycles

0% 10% 20% 30% 40% 50% 60% 70% 429.mcf Graph analytics SVM MongoDB

% of cpu cycles spent by page walk

Virtual address Physical address Page table

slide-3
SLIDE 3

High address translation cost

3

Cpu cycles

0% 10% 20% 30% 40% 50% 60% 70% 429.mcf Graph analytics SVM MongoDB

Guest page table walk Host page table walk

% of cpu cycles spent by page walk

Virtual address Guest physical address Host physical address Guest page table Host page table

  • Virtualization requires additional address translation
slide-4
SLIDE 4

Huge pages improve TLB coverage

  • Architecture supports larger page size (e.g., 2MB page)
  • Intel: 0 to 1,536 entries in 2 years (2013 ~ 2015)
  • Operating system has the burden of better huge page support

4

0% 1% 2% 3% 4% 5% Sandy Bridge Ivy Bridge Haswell Skylake

4KB page 2MB page

TLB coverage proportional to 64 GB DRAM

0.11% 0.05% 4.6% 0.01% 0.01% 0.1% 3.2% 0.1% 2013 2015 2014 2011

slide-5
SLIDE 5

Operating system support for huge pages

  • OS transparently allocates/deallocates huge pages
  • Huge pages in both guest and host

5

Linux FreeBSD LWN.net, 2011

slide-6
SLIDE 6

Huge pages improve performance

  • Application speed up over using base pages only

6

Speed up 0% 10% 20% 30% 40% 50% 60% 4 2 9 . m c f 
 ( S p e c C P U ) C a n n e a l 
 ( P A R S E C ) S V M 
 ( L i b l i n e a r ) G r a p h a n a l y t i c s 
 ( P

  • w

e r G r a p h ) M a c h i n e l e a r n i n g 
 ( S p a r k M L l i b ) W e b s e r v e r 
 ( C l

  • u

d s t

  • n

e ) R e d i s M

  • n

g

  • D

B

B e t t e r Average

slide-7
SLIDE 7

Are huge pages a free lunch?

7

slide-8
SLIDE 8

Are huge pages a free lunch?

8

slide-9
SLIDE 9

Are huge pages a free lunch?

8

slide-10
SLIDE 10

Are huge pages a free lunch?

8

slide-11
SLIDE 11

Are huge pages a free lunch?

8

slide-12
SLIDE 12

Huge page pathologies in Linux

  • High page fault latency
  • Memory bloating
  • Unfair huge page allocation
  • Uncoordinated memory management

9

slide-13
SLIDE 13

Huge page pathologies in Linux

  • High page fault latency
  • Memory bloating
  • Unfair huge page allocation
  • Uncoordinated memory management

10

slide-14
SLIDE 14

Ingens

Efficient huge page management system

11

Linux Ingens

Synchronous allocation Asynchronous allocation Greedy allocation Spatial utilization based allocation How to allocate huge pages?

Problems

High page fault latency Memory bloating

slide-15
SLIDE 15

High page fault latency

12

slide-16
SLIDE 16

Huge page allocation increases page fault latency

  • Page allocation path of both base and huge page

13

Allocate page(s) Get page(s) from free page list Zero the page(s) Map the page(s) to page table Page fault handler Physical memory manager Application pause Application resume

Page fault latency

  • 4KB page : 3.6 us
  • 2MB page : 378.0 us (mostly from page zeroing)
  • Increases tail latency
slide-17
SLIDE 17

Huge page allocation might require extra memory copying

  • Page allocation path of huge page

14

Get page(s) from free page list Zero the page(s) Map the page(s) to page table Page fault handler Physical memory manager Application pause Application resume Allocate page(s)

slide-18
SLIDE 18

Huge page allocation might require extra memory copying

  • Page allocation path of huge page

14

Get page(s) from free page list Zero the page(s) Map the page(s) to page table Page fault handler Physical memory manager Not enough contiguous memory Application pause Application resume Allocate page(s)

slide-19
SLIDE 19

External fragmentation

15

Not enough contiguous memory

slide-20
SLIDE 20

External fragmentation

15

Virtual address Physical address

Huge page boundary

B B B B B B

Allocated Base page

Not enough contiguous memory

B

  • As system ages, physical memory is

fragmented

  • 2 minutes to fragment 24 GB
  • All memory sizes eventually fragment
  • Linux compacts physical memory to

create contiguous pages

slide-21
SLIDE 21

External fragmentation

15

Virtual address Physical address

Huge page boundary

B B B B

Allocated Base page

Not enough contiguous memory

B B

  • As system ages, physical memory is

fragmented

  • 2 minutes to fragment 24 GB
  • All memory sizes eventually fragment
  • Linux compacts physical memory to

create contiguous pages

slide-22
SLIDE 22

External fragmentation

  • As system ages, physical memory is

fragmented

  • 2 minutes to fragment 24 GB
  • All memory sizes eventually fragment
  • Linux compacts physical memory to

create contiguous pages

Virtual address Physical address

Huge page boundary

B B B B B B

Allocated Base page

Not enough contiguous memory

H

slide-23
SLIDE 23

External fragmentation

Not enough contiguous memory

slide-24
SLIDE 24

Huge page allocation might require extra memory copying

  • Page allocation path of huge page includes memory compaction

17

Get page(s) from free page list Zero the page(s) Map the page(s) to page table Page fault handler Physical memory manager Not enough contiguous memory Application pause Application resume Allocate page(s)

slide-25
SLIDE 25

Huge page allocation might require extra memory copying

  • Page allocation path of huge page includes memory compaction

17

Get page(s) from free page list Zero the page(s) Map the page(s) to page table Page fault handler Physical memory manager Not enough contiguous memory Compact physical memory Application pause Application resume Allocate page(s)

slide-26
SLIDE 26

Huge page allocation might require extra memory copying

  • Page allocation path of huge page includes memory compaction

17

Get page(s) from free page list Zero the page(s) Map the page(s) to page table Page fault handler Physical memory manager Not enough contiguous memory Compact physical memory Compaction may

  • r may not succeed

Application pause Application resume Allocate page(s)

slide-27
SLIDE 27

Ingens: asynchronous allocation

18

Page fault handler Asynchronous promotion

  • Page fault handler only

allocates base pages

  • Huge page allocation in

background

  • Memory compaction in

background

  • No extra page fault latency
  • No huge page zeroing
  • No compaction

bit vector 1 bit per base page Read/update

  • n each base page fault

Promotion Kernel thread

Fast page fault handling

slide-28
SLIDE 28

Page fault latency experiment

  • Machine specification
  • Two Intel Xeon E5-2640 2.60GHz CPUs
  • 64GB memory and two 250 MB SSDs
  • Cloudstone workload (latency sensitive)
  • Web service for social event planning
  • nginx/PHP/MySQL running in virtual machines
  • 85% read, 10% login, 5% write workloads
  • 2 of 7 web pages modified to use modern web page sizes
  • The average web page is 2.1 MB

https://www.soasta.com/blog/page-bloat-average-web-page-2-mb/

19

slide-29
SLIDE 29

Cloudstone result

  • Memory is highly fragmented
  • Ingens reduces
  • average latency up to 29.2%
  • tail latency up to 41.4%
  • Linux page fault handler

performs 461,383 memory compactions

20

Linux Ingens 922.3 1091.9 (+18%)

Throughput (requests/s) Latency (millisecond)

100 200 300 400 500 600 Avg. 90th Avg. 90th

Linux Ingens

View event Visit home page

B e t t e r

slide-30
SLIDE 30

Cloudstone result

  • Memory is highly fragmented
  • Ingens reduces
  • average latency up to 29.2%
  • tail latency up to 41.4%
  • Linux page fault handler

performs 461,383 memory compactions

20

Linux Ingens 922.3 1091.9 (+18%)

Throughput (requests/s) Latency (millisecond)

100 200 300 400 500 600 Avg. 90th Avg. 90th

Linux Ingens

View event Visit home page

B e t t e r

slide-31
SLIDE 31

Memory bloating

21

Application occupies more memory than it uses

slide-32
SLIDE 32

Internal fragmentation

  • Greedy allocation in Linux
  • Allocate a huge page on first

fault to huge page region

  • The huge page region may not

be fully used

  • Greedy allocation causes severe

internal fragmentation

  • Memory use often sparse

22

Virtual address Physical address

Huge page boundary

H H

Used virtual address Unused virtual address Huge page region

H

slide-33
SLIDE 33

Memory bloating experiment

  • Redis
  • Delete 70% objects after

populating 8KB objects

  • MongoDB
  • 15 million get requests for

1KB object with YCSB

23

Using huge page Using only base page Redis 20.7GB (+69%) 12.2GB MongoDB 12.4GB (+23%) 10.1GB

Physical memory consumption

Bloating makes memory consumption unpredictable Memory-intensive applications can’t provision to avoid swap

slide-34
SLIDE 34

Ingens: Spatial utilization based allocation

  • Ingens monitors spatial utilization
  • f each huge page region
  • Utilization-based allocation
  • Page fault handler requests

promotion when the utilization is beyond a threshold (e.g., 90%)

  • Bounds the size of internal

fragmentation

24

Virtual address Physical address

H B B B B

100% utilization 75% utilization 25% utilization

slide-35
SLIDE 35

Redis memory bloating experiment

25

Physical memory consumption GET throughput

12.2 GB Linux (base only) 20.7 GB Linux (huge) Ingens Linux (base only) Linux (huge) 21.7K 19.0K Ingens 12.3 GB 20.9K

  • 4%

+ 10% Better Better

Huge : 2MB page Base : 4KB page

slide-36
SLIDE 36

Ingens overhead

  • Overhead for memory intensive application
  • Overhead for non-memory intensive application

26

429.mcf Graph Spark Canneal SVM Redis MongoDB

0.9% 0.9% 0.6% 1.9% 1.3% 0.2% 0.6%

Kernel build Grep Parsec 3.0 Benchmark

0.2% 0.4% 0.8%

Ingens overhead is negligible

slide-37
SLIDE 37

27

Linux Ingens

Synchronous allocation Asynchronous allocation Greedy allocation Spatial utilization based allocation

Ingens

Make huge pages widely used in practice

Source code is available at https://github.com/ut-osa/ingens

Advantages

No extra page fault latency Bound memory bloating

slide-38
SLIDE 38

Backup slides

28

slide-39
SLIDE 39

Other operating systems

  • Window, MacOS
  • Does not support transparent huge page
  • FreeBSD
  • Very conservative approach
  • No memory compaction functionality
  • Performance speedup in Linux and FreeBSD

29

SVM Canneal Redis FreeBSD 1.28 1.13 1.02 Linux 1.30 1.21 1.15 Ingens 1.29 1.19 1.15

slide-40
SLIDE 40
  • User-controlled huge page management
  • Admin reserves huge page in advance
  • New APIs for memory allocation/deallocation
  • It could fail to reserve huge pages when memory is

fragmented

  • Transparent huge page management
  • Developers do not know about huge page
  • OS Transparently allocates/deallocates huge pages
  • OS manages memory fragmentation

30

Operating system support for huge pages