iDedup Latency-aware inline deduplication for primary workloads - - PowerPoint PPT Presentation

▶
idedup
SMART_READER_LITE
LIVE PREVIEW

iDedup Latency-aware inline deduplication for primary workloads - - PowerPoint PPT Presentation

Aug 11, 2023 315 likes •609 views

iDedup Latency-aware inline deduplication for primary workloads Kiran Srinivasan, Tim Bisson Garth Goodson, Kaladhar Voruganti Advanced Technology Group NetApp 1 iDedup overview/context Dedupe exploited effectively here Storage

slide-1
SLIDE 1

Kiran Srinivasan, Tim Bisson Garth Goodson, Kaladhar Voruganti Advanced Technology Group NetApp

iDedup

Latency-aware inline deduplication for primary workloads

1

slide-2
SLIDE 2

iDedup – overview/context

2

NFS/CIFS/iSCSI

Primary Storage Storage Clients Secondary Storage

NDMP/Other

Primary Storage:

  • Performance & reliability are key features
  • RPC-based protocols => latency sensitive
  • Only offline dedupe techniques developed

Dedupe exploited effectively here => 90+% savings

iDedup

  • Inline/foreground dedupe technique for primary
  • Minimal impact on latency-sensitive workloads
slide-3
SLIDE 3

3

Dedupe techniques (offline vs inline)

Offline dedupe

  • First copy on stable storage is not deduped
  • Dedupe is a post-processing/background activity

Inline dedupe

  • Dedupe before first copy on stable storage
  • Primary => Latency should not be affected!
  • Secondary => Dedupe at ingest rate (IOPS)!
slide-4
SLIDE 4

Why inline dedupe for primary?

4

Provisioning/Planning is easier

  • Dedupe savings are seen right away
  • Planning is simpler as capacity values are accurate

No post-processing activities

  • No scheduling of background processes
  • No interference => front-end workloads are not affected
  • Key for storage system users with limited maintenance windows

Efficient use of resources

  • Efficient use of I/O bandwidth (offline has both reads + writes)
  • File system’s buffer cache is more efficient (deduped)

Performance challenges have been the key obstacle

  • Overheads (CPU + I/Os) for both reads and writes hurt latency
slide-5
SLIDE 5

iDedup – Key features

5

Minimizes inline dedupe performance overheads

  • Leverages workload characteristics
  • Eliminates almost all extra I/Os due to dedupe processing
  • CPU overheads are minimal

Tunable tradeoff: dedupe savings vs performance

  • Selective dedupe => some loss in dedupe capacity savings

iDedup can be combined with offline techniques

  • Maintains same on-disk data structures as normal file system
  • Offline dedupe can be run optionally
slide-6
SLIDE 6

Related Work

6

Workload/ Method Offline Inline Primary NetApp ASIS EMC Celerra IBM StorageTank

iDedup

Secondary (No motivation for systems in this category) EMC DDFS, EMC Cluster DeepStore, NEC HydraStor, Venti, SiLo, Sparse Indexing, ChunkStash, Foundation, Symantec, EMC Centera

zFS*

slide-7
SLIDE 7

Outline

¡ Inline dedupe challenges ¡ Our Approach ¡ Design/Implementation details ¡ Evaluation results ¡ Summary

7

slide-8
SLIDE 8

Fragmentation with random seeks

Inline Dedupe - Read Path challenges

8

Inherently, dedupe causes disk-level fragmentation !

  • Sequential reads turn random => more seeks => more latency
  • RPC based protocols (CIFS/NFS/iSCI) are latency sensitive
  • Dataset/workload property

Primary workloads are typically read-intensive

  • Usually the Read/Write ratio is ~70/30
  • Inline dedupe must not affect read performance !
slide-9
SLIDE 9

Inline Dedupe – Write Path Challenges

9

CPU overheads in the critical write path

  • Dedupe requires computing the Fingerprint (Hash) of each block
  • Dedupe algorithm requires extra cycles

Client Write Write Logging Compute Hash Dedupe Algorithm Write Allocation Disk IO De-stage phase

Extra random I/Os in the write path due to dedupe algorithm

  • Dedupe metadata (Finger Print DB) lookups and updates
  • Updating the reference counts of blocks on disk

Dedupe metadata

Random Seeks !!

slide-10
SLIDE 10

10

Our Approach

slide-11
SLIDE 11

iDedup – Intuition

11

Is there a good tradeoff between capacity savings and latency performance?

slide-12
SLIDE 12

iDedup – Solution to Read Path issues

12

Insight 1: Dedupe only sequences of disk blocks

  • Solves fragmentation => amortized seeks during reads
  • Selective dedupe, leverages spatial locality
  • Configurable minimum sequence length

Fragmentation with random seeks Sequences, with amortized seeks

slide-13
SLIDE 13

iDedup – Write Path issues

13

How can we reduce dedupe metadata I/Os? Flash(?) - read I/Os are cheap, but frequent updates are expensive

slide-14
SLIDE 14

iDedup – Solution to Write Path issues

14

Insight 2: Keep a smaller dedupe metadata as an in-memory cache

  • No extra IOs
  • Leverages temporal locality characteristics in duplication
  • Some loss in dedupe (a subset of blocks are used)

Time Cached fingerprints Blocks

slide-15
SLIDE 15

iDedup - Viability

15

Dedup Ratio

Original Spatial Locality Spatial + Temporal Locality

Is this loss in dedupe savings Ok? Both spatial and temporal localities are dataset/workload properties ! ⇒ Viable for some important primary workloads

slide-16
SLIDE 16

16

Design and Implementation

slide-17
SLIDE 17

iDedup Architecture

17

NVRAM (Write log blocks) iDedup Algorithm I/Os (Reads + Writes)

Dedupe metadata (FPDB)

Disk De-stage File system (WAFL)

slide-18
SLIDE 18

iDedup - Two key tunable parameters

18

Minimum sequence length – Threshold

  • Minimum number of sequential duplicate blocks on disk
  • Dataset property => ideally set to expected fragmentation
  • Different from larger block size – variable vs fixed
  • Knob between performance (fragmentation) and dedupe

Dedupe metadata (Fingerprint DB) cache size

  • Workload’s working set property
  • Increase in cache size => decrease in buffer cache
  • Knob between performance (cache hit ratio) and dedupe
slide-19
SLIDE 19

iDedup Algorithm

The iDedup algorithm works in 4 phases for every file: ¡ Phase 1 (per file):Identify blocks for iDedup

– Only full, pure data blocks are processed – Metadata blocks, special files ignored

¡ Phase 2 (per file) : Sequence processing

– Uses the dedupe metadata cache – Keeps track of multiple sequences

¡ Phase 3 (per sequence): Sequence pruning

– Eliminate short sequences below threshold – Pick among overlapping sequences via a heuristic

¡ Phase 4 (per sequence): Deduplication of sequence

19

slide-20
SLIDE 20

20

Evaluation

slide-21
SLIDE 21

Evaluation Setup

¡ NetApp FAS 3070, 8GB RAM, 3 disk RAID0 ¡ Evaluated by replaying real-world CIFS traces

– Corporate filer traces in NetApp DC (2007)

¡ Read data: 204GB (69%), Write data: 93GB

– Engineering filer traces in NetApp DC (2007)

¡ Read data: 192GB (67%), Write data: 92GB

¡ Comparison points

– Baseline: System with no iDedup – Threshold-1: System with full dedupe (1 block)

¡ Dedupe metadata cache: 0.25, 0.5 & 1GB

21

slide-22
SLIDE 22

8 10 12 14 16 18 20 22 24 1 2 4 8 Deduplication ratio (%) Threshold .25 GB .5 GB 1 GB

Results: Deduplication ratio vs Threshold

22

Dedupe ratio vs Thresholds, Cache sizes (Corp)

Less than linear decrease in dedupe savings ⇒ Spatial locality in dedupe Ideal Threshold = Biggest threshold with least decrease in dedupe savings ⇒ Threshold-4 ⇒ ~60% of max

slide-23
SLIDE 23

Results: Disk Fragmentation (req sizes)

23

10 20 30 40 50 60 70 80 90 100 1 8 16 24 32 40 Percentage of Total Requests Request Sequence Size (Blocks) Baseline (Mean=15.8) Threshold-1 (Mean=12.5) Threshold-2 (Mean=14.8) Threshold-4 (Mean=14.9) Threshold-8 (Mean=15.4)

Fragmentation for

  • ther thresholds are

between Baseline and Thresh-1 ⇒ Tunable fragmentation

Least fragmentation Max fragmentation

CDF of block request sizes (Engg, 1GB)

slide-24
SLIDE 24

Results: CPU Utilization

24

10 20 30 40 50 60 70 80 90 100 5 10 15 20 25 30 35 40 Percentage of CPU Samples CPU Utilization (%) Baseline (Mean=13.2%) Threshold-1 (Mean=15.0%) Threshold-4 (Mean=16.6%) Threshold-8 (Mean=17.1%)

CDF of CPU utilization samples (Corp, 1GB)

Larger variance (long tail) compared to baseline ⇒ But, mean difference is less than 4%

slide-25
SLIDE 25

Results: Latency Impact

25

75 80 85 90 95 100 5 10 15 20 25 30 Percentage of Requests Response Time (ms) Baseline Threshold-1 Threshold-8

CDF of client response time (Corp, 1GB)

Affects >2ms

Latency impact for longer response times ( > 2ms) ⇒ Thresh-1 mean latency affected by ~13% vs baseline ⇒ Diff between Thresh-8 and baseline <4%!

slide-26
SLIDE 26

Summary

¡ Inline dedupe has significant performance challenges

– Reads : Fragmentation, Writes: CPU + Extra I/Os

¡ iDedup creates tradeoffs b/n savings and performance

– Leverage dedupe locality properties – Avoid fragmentation – dedupe only sequences – Avoid extra I/Os – keep dedupe metadata in memory

¡ Experiments for latency-sensitive primary workloads

– Low CPU impact – < 5% on the average – ~60% of max dedupe, ~4% impact on latency

¡ Future work: Dynamic threshold, more traces ¡ Our traces are available for research purposes

26

slide-27
SLIDE 27

Acknowledgements

¡ NetApp WAFL Team

– Blake Lewis, Ling Zheng, Craig Johnston, Subbu PVS, Praveen K, Sriram Venketaraman

¡ NetApp ATG Team

– Scott Dawkins, Jeff Heller

¡ Shepherd

– John Bent

¡ Our Intern (from UCSC)

– Stephanie Jones

27

slide-28
SLIDE 28

28