I. Cano*, S. Aiyar, V. Arora, M. Bhattacharyya, A. Chaganti, C. - PowerPoint PPT Presentation

I. Cano*, S. Aiyar, V. Arora, M. Bhattacharyya, A. Chaganti, C. Cheah, B. Chun, K. Gupta, V. Khot and A. Krishnamurthy* Nutanix Inc. *University of Washington NSDI ’17

• Transparent access to storage • Scale up storage by buying more resources

• Automatic replication and recovery • Seamless integration of SSDs and HDDs • Snapshotting and reclamation of unnecessary data • Space-saving transformations • …

inside • Co-design Reinforcement Learning

inside • Co-design Reinforcement Learning

I/O SERVICE STORAGE MGMT (foreground) (background) KEY-VALUE STORE

• extents extent groups file 0-8K 8-16K 16-24K 24-32K 32-40K 33 44 56 Extent Id Map eid egid Extent Group Id Map egid physical location 33 1984 1984 disk1,disk2 44 1984 … … 56 1984

STORAGE MGMT I/O SERVICE uses MapReduce KEY-VALUE STORE

Metadata Ring Node A Curator (A) Slave Node D Node B Curator (D) Curator (B) Slave Master Curator (C) Slave Node C

Curator (A) 120 -> mtime Curator (D) Curator (B) 120 -> atime Curator (C) (120,mtime,atime)

• Extensible, flexible and scalable framework

Tasks I/O SERVICE STORAGE MGMT KEY-VALUE STORE

1 Fraction of Clusters 50% of clusters 0.8 have 75% usage 0.6 0.4 0.2 0 0 10 20 30 40 50 60 70 80 90 100 SSD Usage (%)

• Extensible, flexible and scalable framework • Synchronization between background tasks and foreground I/O

AGENT State s t Reward r t Action a t r t+1 ENVIRONMENT s t+1

40 METRIC AVG IMPROVEMENT 35 LATENCY 12 % 30 Improvement (%) SSD READS 16 % 25 20 15 10 5 0 oltp oltp skewed oltp varying oltp and vdi dss latency ssd reads

Curator inside • Co-designed Reinforcement Learning