HashCache: Cache Storage for the Next Billion Anirudh Badam - - PowerPoint PPT Presentation

HashCache: Cache Storage for the Next Billion

1

Anirudh Badam KyoungSoo Park Vivek S. Pai Larry L. Peterson

Princeton University

Next Billion Internet Users

2

Next Billion Internet Users

• Schools, urban middle class in

developing regions

2

Next Billion Internet Users

• Schools, urban middle class in

developing regions

• Affordable hardware
• OLPC and Intel Classmate

2

$200

Next Billion Internet Users

• Schools, urban middle class in

developing regions

• Affordable hardware
• OLPC and Intel Classmate
• Expensive Internet
• $1500+ per month per Mbps
• Unlikely to improve in the

near future

2

$200 $1500 per month

Bandwidth Saving

3

Bandwidth Saving

• Connectivity is a precious

resource

3

Bandwidth Saving

• Connectivity is a precious

resource

• Saving bandwidth important
• Disk caches reduce network

bandwidth requirement

3

Internet

Cache

Bandwidth Saving

• Connectivity is a precious

resource

• Saving bandwidth important
• Disk caches reduce network

bandwidth requirement

• Good news - disk is very

cheap

• $100/TB

3

Internet

Cache

Why Large Caches?

4

Why Large Caches?

• Larger bandwidth savings
• Refreshes cheaper than re-fetches
• Overnight prefetch, content push from peers

4

Why Large Caches?

• Larger bandwidth savings
• Refreshes cheaper than re-fetches
• Overnight prefetch, content push from peers
• Good offline behavior
• Preload websites
• Enables local search

4

Why Large Caches?

• Larger bandwidth savings
• Refreshes cheaper than re-fetches
• Overnight prefetch, content push from peers
• Good offline behavior
• Preload websites
• Enables local search
• Save even on dynamic content
• WAN Acceleration = packet caching

4

What is the Cost?

5

What is the Cost?

• In-memory data structures
• Hash table avoids seeks for misses
• Cache replacement (LRU, etc)

5

70 seeks per sec

What is the Cost?

• In-memory data structures
• Hash table avoids seeks for misses
• Cache replacement (LRU, etc)
• RAM index size per TB
• Open Source (Squid) - 10 GB
• Commercial (Tiger) - 5 GB

5

70 seeks per sec

What is the Cost?

• In-memory data structures
• Hash table avoids seeks for misses
• Cache replacement (LRU, etc)
• RAM index size per TB
• Open Source (Squid) - 10 GB
• Commercial (Tiger) - 5 GB
• Can not use laptops for cache
• 2 servers, $2K each = 20 laptops

5

70 seeks per sec

x 20

Our Solution

6

• HashCache: storage engine w/ plug-in indexing
• 6 schemes in paper, 3 shown here
• New Web proxy using HashCache engine

Our Solution

6

• HashCache: storage engine w/ plug-in indexing
• 6 schemes in paper, 3 shown here
• New Web proxy using HashCache engine
• Efficiency and Performance:
• Range of policies trading speed and memory
• 20-50x less RAM for Squid-like speed
• 6-10x less RAM vs Tiger (Commercial)

Our Solution

6

• HashCache: storage engine w/ plug-in indexing
• 6 schemes in paper, 3 shown here
• New Web proxy using HashCache engine
• Efficiency and Performance:
• Range of policies trading speed and memory
• 20-50x less RAM for Squid-like speed
• 6-10x less RAM vs Tiger (Commercial)
• Can now use cheap laptops vs servers
• Even for TB-sized caches

Our Solution

6

Performance: req/sec/disk

1 2 3 4 70 140 210 280 350 HashCache Current Squid Tiger

Gigabytes/Dollar

Our Solution

6

Better

A Brief History of Cache

7

A Brief History of Cache

7

URL

A Brief History of Cache

7

URL ddddhhhhbbbb

A Brief History of Cache

7

URL ddddhhhhbbbb

} } }

Directory Level 1 Directory Level 2 Actual File

A Brief History of Cache

7

URL ddddhhhhbbbb

} } }

Directory Level 1 Directory Level 2 Actual File

In-memory Hashtable

A Brief History of Cache

7

URL ddddhhhhbbbb

} } }

Directory Level 1 Directory Level 2 Actual File

In-memory Hashtable

Squid

A Brief History of Cache

7

URL

A Brief History of Cache

7

URL Filesystem

Circular Log

A Brief History of Cache

7

URL hash_value Filesystem

Circular Log

A Brief History of Cache

7

URL hash_value

• ffset

Memory hash_value Filesystem

Circular Log

A Brief History of Cache

7

URL hash_value

• ffset

Memory hash_value Filesystem

Circular Log

Tiger

A Brief History of Cache

• Open Source Implementation - Squid
• Multiple seeks for hit, miss and write
• Dependent on default filesystems
• Commercial/High Performance - Tiger
• One seek for hit
• Custom file layout

7

Index Element Sizes

8

Functionality Existence Identification Replacement Policy Location Information Other

Index Element Sizes

8

Functionality Implementation Choice Existence Identification Hashtable Chaining Pointers Hash Replacement Policy LRU List Pointers Location Information Disk Offset, Version Number, etc Other Expiration Date, Size, HTTP header info etc Total

Index Element Sizes

8

Functionality Implementation Choice Existence Identification Hashtable Chaining Pointers Hash Replacement Policy LRU List Pointers Location Information Disk Offset, Version Number, etc Other Expiration Date, Size, HTTP header info etc Total

Index Element Sizes

Focusing mainly

• n reducing the

size of the index

8

Functionality Implementation Choice Squid (Bits) Existence Identification Hashtable Chaining Pointers 96 Hash 160 Replacement Policy LRU List Pointers 64 Location Information Disk Offset, Version Number, etc Other Expiration Date, Size, HTTP header info etc 240 Total 560

Index Element Sizes

8

Functionality Implementation Choice Squid (Bits) Tiger (Bits) Existence Identification Hashtable Chaining Pointers 96 96 Hash 160 32 Replacement Policy LRU List Pointers 64 64 Location Information Disk Offset, Version Number, etc 40 Other Expiration Date, Size, HTTP header info etc 240 Total 560 232

Index Element Sizes

8

Revisiting the Index...

9

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem

9

Cache size limited by memory size

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem

9

Cache size limited by memory size

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem Performance depends on disk seeks

9

Cache size limited by memory size

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem Performance depends on disk seeks

reduce the dependency

9

Cache size limited by memory size

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem Performance depends on disk seeks

reduce the dependency

• ptimize

for seeks

9

Revisiting the Index...

• Eliminate(?) in-memory index

In-memory Index

Application Cache Manager Reliable Filesystem

9

Revisiting the Index...

• Eliminate(?) in-memory index
• Need membership and location

information

In-memory Index

Application Cache Manager Reliable Filesystem

9

Revisiting the Index...

• Eliminate(?) in-memory index
• Need membership and location

information

• Use disk as hash table

In-memory Index

Application Cache Manager Reliable Filesystem

9

Revisiting the Index...

• Eliminate(?) in-memory index
• Need membership and location

information

• Use disk as hash table
• On disk data structures for

key lookup

• Store the object as values for

the keys

In-memory Index

Application Cache Manager Reliable Filesystem

9

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem HashCache Engine

9

Application Logic Application Cache Manager Reliable Filesystem

In-memory Index

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem HashCache Engine

9

Application Logic Reliable Filesystem

In-memory Index

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem HashCache Engine

9

Application Logic

In-memory Index

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem HashCache Engine

9

Application Logic

Revisiting the Index...

In-memory Index

Application Cache Manager Reliable Filesystem HashCache Engine

9

Application Logic

HashCache: Basic Policy

10

URL

HashCache: Basic Policy

10

URL

HashCache: Basic Policy

data

10

URL

HashCache: Basic Policy

hash_value

data

H Bits

10

URL

HashCache: Basic Policy

hash_value

data

H Bits

N contiguous blocks

Filesystem

(Disk Table)

10

URL

HashCache: Basic Policy

hash_value

% N

t

data

H Bits

N contiguous blocks

Filesystem

(Disk Table)

10

URL

HashCache: Basic Policy

hash_value

% N

t

data tth block

H Bits

N contiguous blocks

Filesystem

(Disk Table)

10

URL

Circular Log

head

HashCache: Basic Policy

hash_value

% N

t

data tth block

H Bits

N contiguous blocks

Filesystem

(Disk Table)

10

URL

Circular Log

head

HashCache: Basic Policy

hash_value

% N

t

tth block

H Bits

N contiguous blocks

Filesystem

(Disk Table)

10

URL

Circular Log

head

HashCache: Basic Policy

hash_value

% N

t

tth block

H Bits

N contiguous blocks

Filesystem

(Disk Table)

10

• Advantages
• No index memory needed
• Tuned for one seek for most objects

HashCache: Basic Policy

10

• Advantages
• No index memory needed
• Tuned for one seek for most objects
• Disadvantages
• One seek per miss
• No collison control
• No cache replacement policy

HashCache: Basic Policy

10

Collision Control

11

Collision Control

11

Collision Control

Chaining

11

Collision Control

• Does not transition well

to disk-based

• Multiple seeks per
• peration
• Walking hash bin list
• Global replacement

policy crosses bins

Chaining

11

Collision Control

Chaining

11

Collision Control

• Fixed locations where each
• bject can be found
• Allocated contiguously,

read together

Set Associativity T

• Ways

11

Collision Control

• Fixed locations where each
• bject can be found
• Allocated contiguously,

read together

• Seek time dominates short

read

Set Associativity T

• Ways

11

Collision Control

• Fixed locations where each
• bject can be found
• Allocated contiguously,

read together

• Seek time dominates short

read

• Eliminate global cache

replacement policies

Set Associativity T

• Ways

11

Reducing Seeks

12

Reducing Seeks

• In-memory hash table
• Too much memory for

pointers

Bin Pointers 32 Chaining Pointers 64 Hash 32 Total (bits) 128

In-memory Hash Table

12

Reducing Seeks

• In-memory hash table
• Too much memory for

pointers

• Disk is already a hash table
• Pointers not needed
• Large bitmap with the

same layout as the disk

Disk Table

12

Reducing Seeks

• In-memory hash table
• Too much memory for

pointers

• Disk is already a hash table
• Pointers not needed
• Large bitmap with the

same layout as the disk

• Just store hash per URL

Disk Table

In-memory Bitmap

H Bits Disk Block

12

Reducing Seeks

• In-memory hash table
• Too much memory for

pointers

• Disk is already a hash table
• Pointers not needed
• Large bitmap with the

same layout as the disk

• Just store hash per URL

Disk Table

In-memory Bitmap

H Bits Disk Block

12

Reducing Seeks

• Original hash of the URL:

64 bits

12

64

Original Hash

Reducing Seeks

• Original hash of the URL:

64 bits

• Eliminate bits for (same)

bin # (228 objs, 8-way, #bins=225 (S))

12

64

Original Hash

39

64 - log(S)

Reducing Seeks

• Original hash of the URL:

64 bits

• Eliminate bits for (same)

bin # (228 objs, 8-way, #bins=225 (S))

• Shrink hash size: Just to

eliminate most false positives (8 bits)

12

64

Original Hash

39

64 - log(S)

8

low FP hash

Cache Replacement

13

64

Original Hash

39

64 - log(S)

8

low FP hash

Cache Replacement

13

• Large disks: 10-100+ million
• bjects
• Global caching relevant when

disk size working set

• When disk >> working set,

local policies global policies

≈ ≈ 64

Original Hash

39

64 - log(S)

8

low FP hash

Cache Replacement

13

• Large disks: 10-100+ million
• bjects
• Global caching relevant when

disk size working set

• When disk >> working set,

local policies global policies

• Local replacement benefits
• 3 bits per URL
• Performed on contiguous objects
• False positives limited by set size

≈ ≈ 64

Original Hash

39

64 - log(S)

8

low FP hash

11

hash + rank

HashCache: SetMem Policy

14

HashCache: SetMem Policy

URL

14

HashCache: SetMem Policy

URL

data

14

HashCache: SetMem Policy

URL

hash_value

% S

t

data

14

HashCache: SetMem Policy

URL

hash_value

% S

t

data

Filesystem

head

Memory

14

HashCache: SetMem Policy

URL

hash_value

% S

t

data

Filesystem

head

Memory

11 Bits tth set tth set

14

HashCache: SetMem Policy

URL

hash_value

% S

t

data

Filesystem

head

Memory

11 Bits tth set tth set

14

HashCache: SetMem Policy

URL

hash_value

% S

t

Filesystem

head

Memory

11 Bits tth set tth set

14

HashCache: SetMem Policy

URL

hash_value

% S

t

LRU LRU

Filesystem

head

Memory

11 Bits tth set tth set

14

• Advantages
• No seeks for most misses
• 1 seek per read, 1 seek per write
• Good hash + replacement in 11 bits

HashCache: SetMem Policy

14

• Advantages
• No seeks for most misses
• 1 seek per read, 1 seek per write
• Good hash + replacement in 11 bits
• Disadvantages
• Writes still need seeks

HashCache: SetMem Policy

14

Further Reducing Seeks

Disk Table

15

Further Reducing Seeks

• Storing objects by hash can

produce random reads & writes

Disk Table

15

Further Reducing Seeks

• Storing objects by hash can

produce random reads & writes

• Restructure on-disk table

Disk Table

15

Further Reducing Seeks

• Storing objects by hash can

produce random reads & writes

• Restructure on-disk table
• Store only hash, rank, offset

15

11 hash

rank

Further Reducing Seeks

• Storing objects by hash can

produce random reads & writes

• Restructure on-disk table
• Store only hash, rank, offset
• Move all data to log

15

11 hash

rank

43 hash

rank

• ffset

Further Reducing Seeks

• Storing objects by hash can

produce random reads & writes

• Restructure on-disk table
• Store only hash, rank, offset
• Move all data to log
• Benefits
• Group writes amortize seeks
• Scheduling related writes

enables read prefetch

• Both reads & writes < 1 seek

15

11 hash

rank

43 hash

rank

• ffset

Further Reducing Seeks

• Storing objects by hash can

produce random reads & writes

• Restructure on-disk table
• Store only hash, rank, offset
• Move all data to log
• Benefits
• Group writes amortize seeks
• Scheduling related writes

enables read prefetch

• Both reads & writes < 1 seek

15

HC-Basic

43

HC-Log

11 HC-SetMem

Further Reducing Seeks

• Storing objects by hash can

produce random reads & writes

• Restructure on-disk table
• Store only hash, rank, offset
• Move all data to log
• Benefits
• Group writes amortize seeks
• Scheduling related writes

enables read prefetch

• Both reads & writes < 1 seek

15

560

Squid

232

Tiger HC-Basic

43

HC-Log

11 HC-SetMem

HashCache: Log Policy

16

HashCache: Log Policy

URL

16

HashCache: Log Policy

URL

data

16

HashCache: Log Policy

URL

hash_value

% S

t

data

16

HashCache: Log Policy

URL

hash_value

% S

t

data Circular Log

Filesystem

head LRU

Memory

16

HashCache: Log Policy

URL

hash_value

% S

t

data Circular Log

Filesystem

head LRU

Memory

tth set 43 Bits

16

HashCache: Log Policy

URL

hash_value

% S

t

data Circular Log

Filesystem

head LRU

Memory

tth set 43 Bits

16

Implementation

17

Implementation

• HashCache Storage

Engine with plug-in policies

17

Implementation

• HashCache Storage

Engine with plug-in policies

• HashCache Web proxy

using storage engine

17

Implementation

• HashCache Storage

Engine with plug-in policies

• HashCache Web proxy

using storage engine

• Multiple apps on same

box, sharing memory

17

Implementation

• HashCache Storage

Engine with plug-in policies

• HashCache Web proxy

using storage engine

• Multiple apps on same

box, sharing memory

• 20,000 lines C code for

the proxy and 1000 lines for the indexing policies

17

Implementation

• HashCache Storage

Engine with plug-in policies

• HashCache Web proxy

using storage engine

• Multiple apps on same

box, sharing memory

• 20,000 lines C code for

the proxy and 1000 lines for the indexing policies

• Event Driven

implementation with non-blocking I/O

17

Implementation

• HashCache Storage

Engine with plug-in policies

• HashCache Web proxy

using storage engine

• Multiple apps on same

box, sharing memory

• 20,000 lines C code for

the proxy and 1000 lines for the indexing policies

• Event Driven

implementation with non-blocking I/O

• Design similar to that of

Flash Web Server. Helpers for I/O and DNS lookups

17

Implementation

• HashCache Storage

Engine with plug-in policies

• HashCache Web proxy

using storage engine

• Multiple apps on same

box, sharing memory

• 20,000 lines C code for

the proxy and 1000 lines for the indexing policies

• Event Driven

implementation with non-blocking I/O

• Design similar to that of

Flash Web Server. Helpers for I/O and DNS lookups

• Balances load across

multiple disks easily and makes scaling obvious

17

Evaluation - Web Polygraph

• De-facto feature and performance testing tool for web proxies
• Compare all variants of HashCache with Squid and Tiger

18

Evaluation - Web Polygraph

• De-facto feature and performance testing tool for web proxies
• Compare all variants of HashCache with Squid and Tiger

18

Experiment Name Setting Configuration Comparision

Evaluation - Web Polygraph

• De-facto feature and performance testing tool for web proxies
• Compare all variants of HashCache with Squid and Tiger

18

Experiment Name Setting Configuration Comparision Low End

Small School using Laptop

1.4 GHz 256 MB 60 GB SATA HashCache vs Squid vs Tiger

Evaluation - Web Polygraph

• De-facto feature and performance testing tool for web proxies
• Compare all variants of HashCache with Squid and Tiger

18

Experiment Name Setting Configuration Comparision Low End

Small School using Laptop

1.4 GHz 256 MB 60 GB SATA HashCache vs Squid vs Tiger

High End

ISP with High-End Server

2 GHz 3.5 GB 5x18 GB SCSI HashCache-Log vs Squid vsTiger

Evaluation - Web Polygraph

• De-facto feature and performance testing tool for web proxies
• Compare all variants of HashCache with Squid and Tiger

18

Experiment Name Setting Configuration Comparision Low End

Small School using Laptop

1.4 GHz 256 MB 60 GB SATA HashCache vs Squid vs Tiger

High End

ISP with High-End Server

2 GHz 3.5 GB 5x18 GB SCSI HashCache-Log vs Squid vsTiger

Large Disk

Large School with Mini-Tower

1.4 GHz 2 GB 2x1TB USB HashCache-Log vs HashCache-SetMem

15 30 45 60 HC-Basic Squid HC-SetMem HC-Log Tiger

Hit Rate

Max

Hit Rate Comparison

19

Low End Configuration

Low End Configuration

75 150 225 300

Performance req/sec

HC-Basic Squid HC-SetMem Tiger HC-Log

Low End Configuration

75 150 225 300

0% RAM for Index 75% 5% 50% 20%

Open Source and Commercial could index only 18 GB HashCache could index 60 GB

Performance req/sec

HC-Basic Squid HC-SetMem Tiger HC-Log

750 1,500 2,250 3,000

Performance req/sec

Squid Tiger HashCache-Log

High End Configuration

21

5x18 GB Disk

750 1,500 2,250 3,000

Performance req/sec

Squid Tiger HashCache-Log

40% RAM for Index 4% 18%

High End Configuration

21

5x18 GB Disk

Index Efficiency

Squid Tiger HC-Log HC-SetMem 1,500 3,000 4,500 6,000

Index Efficiency

Max Disk for 1GB RAM

Squid Tiger HC-Log HC-SetMem 1,500 3,000 4,500 6,000

Index Efficiency

Max Disk for 1GB RAM

40 reqs/sec 300 300 65

Large Disk Configuration

23

1 TB Disk

75 150 225 300 HC-Basic HC-SetMem HC-Log

Performance req/sec

Large Disk Configuration

23

1 TB Disk

75 150 225 300 HC-Basic HC-SetMem HC-Log

Performance req/sec 150 MB 600 MB 0 MB Index

Conclusions & Status

• New Storage Engine & Web Cache
• From no RAM per object to tiny no. of bits/obj
• 6-10x better than Tiger, 20-50x vs Squid
• Enables large disk w/ only laptop-class machine
• More policies, details in paper
• Suitable for developing-world usage
• Current deployments: Ghana, Nigeria
• Working w/ school supplier on new deployments

24

Thank You!

abadam@cs.princeton.edu

http://tinyurl.com/hashcache

25