Single-pass restore after a media failure Caetano Sauer , Goetz - - PowerPoint PPT Presentation
Single-pass restore after a media failure Caetano Sauer , Goetz Graefe, Theo Hrder 20% of drives fail after 4 years High failure rate on first year (factory defects) Expectation of 50% for 6 years
Caetano Sauer, Goetz Graefe, Theo Härder
2 https://www.backblaze.com/blog/how-long-do-disk-drives-last/
20% of drives fail after 4 years Expectation of 50% for 6 years High failure rate
(factory defects)
3 https://www.backblaze.com/blog/best-hard-drive/
Bad batches among
products More reliability at a premium price tag
4
Sep 2013 Drawbacks of traditional recovery apply (in varying degrees) to all types of media
Log DB Log archive Full Incr. Backups Replacement Primary storage (latency-optimized) Secondary storage (bandwidth-optimized)
5
Scenario:
Sun 11PM Mon 11PM Tue 11PM Wed 11PM Thu 11PM Fri 11PM Sat 11PM
Full Incr. Incr. Incr. Incr. Incr. Incr.
Sun 10PM
Disk failure* * Subject to Murphy‘s law
6
(Assuming 1ms latency of jump-sequential access)
(Assuming 20 log records per modified page)
TOTAL = 45 hours Depends on device Depends on workload (growth rate & skew) and luck Depends on amount
Single-pass restore:
7
8
Fundamental assumption: page-based storage
page = unit of recovery
(i.e., virtually all database systems in use today)
Every log record reflects changes to a delimited region of physical storage
„recovery independence among objects“ (C. Mohan)
9
10
Log DB Log archive Replacement Sorted log archive Full Merge log and backup in a single pass! Merge join
11
12
External sort has two phases: during log archiving (normal processing)
Log DB
during restore
Run Run Run Run Run Run Run Run
Log archive
In-memory sorting
13
15
Run 1 Run 2 Run 3 Run
1000
... Merge Merge Restored DB Full backup Sorted log
No need for incrementals
16
Run 1 Run 2 Run 3 Run
1000
... Merge Merge Restored DB Full backup Buffer Buffer Buffer Buffer RAM
Merge fan-in is limited by memory But number of runs can be kept manageable
17
Back to example: 6 TB with 2% daily change; 20 log records per page
→ Log volume: 2 GB per day; 14 GB per week
→ 20 runs per day
→ 140 MB to merge a whole week worth of log
1 MB of RAM to merge 100 MB of log Runs as history units:
2007
...
2008 Jan 2014 Feb 2014 W 1 2015 W 2 2015
... ...
Mar 2 2015
40 MB to merge 1 year worth of log
(11 monthly + 3 weekly + 6 daily + 20 current day)
19
How often?
Virtual backups
Run Run Run Run Run Run Run Run
Old backup New backup Merge Monthly run
20
Goal: alleviate cost of log replay without overhead of taking a full backup
In single-pass restore:
But does it still make sense?
21
22
i.e., substantially faster media recovery practically for free
i.e., memory is not stolen from buffer pool
i.e., restoring up-to-date DB takes the same as loading an outdated full backup
23
= baseline (no log archiving → no media recovery)
= traditional log archiving (process and copy; no sorting)
= sorting (run generation)
Setup:
24
B = baseline T = traditional log archiving S = sorting (run generation) M = async. merging
Log in DRAM
1.5% Slightly higher CPU utilization
(OLTP workloads rarely fully utilize CPU)
25
B = baseline T = traditional log archiving S = sorting (run generation) M = async. merging
Log in SSD
No observable difference in throughput
26
Small memory footprint, independent
27
(TPC-C databases)
Outdated (days or weeks) Up-to-date
28
30
Log DB
Log archive
In-memory sorting + index load Run Run Run Run Run Run Run Run Backup Restore failed device:
(on buffer pool and healthy/restored storage)
31
http://instantrecovery.github.io
R R Traditional restore: Single-pass restore: Instant restore: New transactions running