SLIDE 1
The Performance Analysis of Cache Architecture based on Alluxio over Virtualized Infrastructure
Xu Chang, Li Zha
1
- Background
- Related Works
- Motivation
- Experiments
- Results
- Conclusion
- Future Work
The Performance Analysis of Cache Architecture based on Alluxio - - PowerPoint PPT Presentation
The Performance Analysis of Cache Architecture based on Alluxio - - PowerPoint PPT Presentation
The Performance Analysis of Cache Architecture based on Alluxio over Virtualized Infrastructure Xu Chang, Li Zha 1 Contents Background Related Works Motivation Experiments Results Conclusion Future Work
SLIDE 2
SLIDE 3
Background
- Cloud Computing
– Computing as a service – Application of resources on demand and payment
- n demand
- Virtualization
– Integrates and encapsulates the resources – Provide the resource in piece – Transparent to users
SLIDE 4
Background
Decoupling vs Traditional Advantage:
- More flexible
- Overall cost is reduced
Shortcoming:
- Performance decline
- Compute Node
Data Node Compute Node Data Node Compute Node Data Node
Traditional Architecture
Decoupling architecture of computing and storage
Compute cluster Compute Node Compute Node Compute Node Data Center (Object Storage) Data Node Data Node Data Node
SLIDE 5
Related Works
For making up the loss of performance
- Traditional optimization method
– Speed up the shuffle part of jobs with SSDs – [kambatla2014truth] [ruan2017improving]
- Reduce the frequency of accessing the object
storage
– Construct the cache layer between applications and
- bject storage
– [shankar2017performance] [qureshi2014cache]
SLIDE 6
Related Works
Alluxio (Tachyon)
- The world’s first memory speed virtual distributed storage
system
- Resides between computation frameworks and storage systems
- Source: https://www.alluxio.org/
SLIDE 7
Motivation
- Only concern about performance, do not care
about cost
- Cost reduction is critical
- Question:
– How to design the caching architecture to make the cost performance highest?
SLIDE 8
Experiments
System architecture
Cloud Storage MapReduce Alluxio MapReduce Alluxio MapReduce Alluxio
- Source: https://www.alluxio.org/
SLIDE 9
Experiments
Experimental environment
Experiment 1: Platform: AWS Servers: m3.2xlarge * 4 Object storage: S3
- Experiment 2:
Platform: G-Cloud Servers: 8 cores & 30G memory * 4 Object storage: Ceph
SLIDE 10
Experiments
Experimental scheme
- Experiment 1:
– Workload: Terasort * 6
- Experiment 2:
– Workload: Hive-Join * 3
- Data Size: 120G
- Cost ratio of memory to SSD
Memory : SSD 8:0 7:1 5:3 3:5 1:7 0:8
SLIDE 11
Results
Experimental 1:
76.00 78.00 80.00 82.00 84.00 86.00 88.00 90.00 92.00
Throughput (MB/s)
Performance
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 100%MEM 87.5%MEM 12.5%SSD 62.5%MEM 37.5%SSD 37.5%MEM 62.5%SSD 12.5%MEM 87.5%SSD 100%SSD COST PERFORMANCE
Cost Performance
SLIDE 12
Results
Experimental 2:
175 180 185 190 195 200 205 210
Throughput (MB/s)
Performance
0.5 1 1.5 2 2.5 3 100%MEM 87.5%MEM 12.5%SSD 62.5%MEM 37.5%SSD 37.5%MEM 62.5%SSD 12.5%MEM 87.5%SSD 100%SSD COST PERFORMANCE
Cost Performance
SLIDE 13
Conclusion
- Hybrid cache architecture is recommended.
- For the workload with large size of output and
small size of hot data, the cost ratio of memory to SSD in cache should be around 1:7
- For the workload with small size of output and
large size of hot data, the cost ratio of memory to SSD in cache should be around 5:3
SLIDE 14
Future Work
- Study several aspects that affect the cost
performance, and try to give a configuration scheme with the best cost performance
- Increase workload types and application
scenarios, so that the conclusion is closer to the real scene and has generality
SLIDE 15