SLIDE 1
UPGRAID Joseph Naps, Ellen Wagner
UPGRAID Joseph Naps, Ellen Wagner
Project Overview
UPGRAID Partition
- RAID Partition
UPGRAID Usage-based striPe replicatinG RAID Joseph Naps, Ellen - - PowerPoint PPT Presentation
UPGRAID Usage-based striPe replicatinG RAID Joseph Naps, Ellen - - PowerPoint PPT Presentation
UPGRAID Joseph Naps, Ellen Wagner UPGRAID Usage-based striPe replicatinG RAID Joseph Naps, Ellen Wagner August 10, 2007 Project Overview UPGRAID Joseph Naps, UPGRAID Partition Ellen Wagner RAID
SLIDE 2
SLIDE 3
UPGRAID Joseph Naps, Ellen Wagner
Project Overview
SLIDE 4
UPGRAID Joseph Naps, Ellen Wagner
Project Overview
SLIDE 5
UPGRAID Joseph Naps, Ellen Wagner
Project Overview
SLIDE 6
UPGRAID Joseph Naps, Ellen Wagner
Project Overview
SLIDE 7
UPGRAID Joseph Naps, Ellen Wagner
Project Overview
SLIDE 8
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
SLIDE 9
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation
SLIDE 10
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation Virtual Machines
SLIDE 11
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation Virtual Machines Modules
SLIDE 12
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation Virtual Machines Modules RAID
SLIDE 13
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation Virtual Machines Modules RAID Reading poorly documented code
SLIDE 14
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation Virtual Machines Modules RAID Reading poorly documented code Properly documenting code
SLIDE 15
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation Virtual Machines Modules RAID Reading poorly documented code Properly documenting code Working with low-level C code
SLIDE 16
UPGRAID Joseph Naps, Ellen Wagner
What We Learned
Kernel Compilation Virtual Machines Modules RAID Reading poorly documented code Properly documenting code Working with low-level C code Block I/Os in Linux
SLIDE 17
UPGRAID Joseph Naps, Ellen Wagner
Approach
SLIDE 18
UPGRAID Joseph Naps, Ellen Wagner
Approach
Read Replication
SLIDE 19
UPGRAID Joseph Naps, Ellen Wagner
Approach
Read Replication Write Replication
SLIDE 20
UPGRAID Joseph Naps, Ellen Wagner
Approach
Read Replication Write Replication Read Indirection
SLIDE 21
UPGRAID Joseph Naps, Ellen Wagner
Approach
Read Replication Write Replication Read Indirection Write Indirection
SLIDE 22
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Replication
SLIDE 23
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Replication
1 UPGRAID determines if the stripe is eligible for replication.
SLIDE 24
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Replication
1 UPGRAID determines if the stripe is eligible for replication. 2 If the stripe is eligible, a read request to the entire stripe is
generated.
SLIDE 25
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Replication
1 UPGRAID determines if the stripe is eligible for replication. 2 If the stripe is eligible, a read request to the entire stripe is
generated.
3 Once that read request completes, a write is generated
and put into a queue to await being sent to an UPGRAID partition.
SLIDE 26
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Replication
SLIDE 27
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Replication
SLIDE 28
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
SLIDE 29
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
1 UPGRAID determines if the stripe is eligible for replication.
SLIDE 30
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
1 UPGRAID determines if the stripe is eligible for replication. 2 If the stripe is eligible a read request to the entire stripe is
generated.
SLIDE 31
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
1 UPGRAID determines if the stripe is eligible for replication. 2 If the stripe is eligible a read request to the entire stripe is
generated.
3 At this point there are sixteen pages (in the page of a
sixty-four KB stripe) with the data from the original stripe.
SLIDE 32
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
1 UPGRAID determines if the stripe is eligible for replication. 2 If the stripe is eligible a read request to the entire stripe is
generated.
3 At this point there are sixteen pages (in the page of a
sixty-four KB stripe) with the data from the original stripe.
4 The data from the original write must now be overlaid on
top of the data read from the stripe to preserve the modifications from the write.
SLIDE 33
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
1 UPGRAID determines if the stripe is eligible for replication. 2 If the stripe is eligible a read request to the entire stripe is
generated.
3 At this point there are sixteen pages (in the page of a
sixty-four KB stripe) with the data from the original stripe.
4 The data from the original write must now be overlaid on
top of the data read from the stripe to preserve the modifications from the write.
5 The modified write is sent to a queue to await submission
to the proper UPGRAID partition.
SLIDE 34
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
SLIDE 35
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
SLIDE 36
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
SLIDE 37
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Replication
SLIDE 38
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Indirection
SLIDE 39
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Indirection
1 UPGRAID determines if the request should be sent to the
RAID5 partition or UPGRAID partition by looking at the head position of each drive. This drive that has the smallest distance to move is chosen to fulfill the request.
SLIDE 40
UPGRAID Joseph Naps, Ellen Wagner
Approach - Read Indirection
1 UPGRAID determines if the request should be sent to the
RAID5 partition or UPGRAID partition by looking at the head position of each drive. This drive that has the smallest distance to move is chosen to fulfill the request.
2 The request is then sent to the appropriate disk and the
application proceeds upon completion of that read request.
SLIDE 41
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Indirection
SLIDE 42
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Indirection
1 The write request to the RAID5 partition is cloned.
SLIDE 43
UPGRAID Joseph Naps, Ellen Wagner
Approach - Write Indirection
1 The write request to the RAID5 partition is cloned. 2 This cloned request gets sent to the appropriate location
- n the UPGRAID partition at the same offset into the
stripe as the original write, thereby preserving the mirroring property between the two stripes.
SLIDE 44
UPGRAID Joseph Naps, Ellen Wagner
Testing Tools
SLIDE 45
UPGRAID Joseph Naps, Ellen Wagner
Testing Tools
Integrity Checker
SLIDE 46
UPGRAID Joseph Naps, Ellen Wagner
Testing Tools
Integrity Checker Workload Profiler
SLIDE 47
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
SLIDE 48
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Automated user level application to test reads and writes to specific blocks.
SLIDE 49
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Automated user level application to test reads and writes to specific blocks. Uses:
SLIDE 50
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Automated user level application to test reads and writes to specific blocks. Uses:
Check and see if data was written to the correct block.
SLIDE 51
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Automated user level application to test reads and writes to specific blocks. Uses:
Check and see if data was written to the correct block. Make sure that modules are performing correctly.
SLIDE 52
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
SLIDE 53
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Proceeds through three testing phases:
SLIDE 54
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Proceeds through three testing phases:
Write Phase
SLIDE 55
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Proceeds through three testing phases:
Write Phase
Generates a write workload across the entire disk space.
SLIDE 56
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Proceeds through three testing phases:
Write Phase
Generates a write workload across the entire disk space.
Read Phase
SLIDE 57
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Proceeds through three testing phases:
Write Phase
Generates a write workload across the entire disk space.
Read Phase
Generates a random read workload across the disk space.
SLIDE 58
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Proceeds through three testing phases:
Write Phase
Generates a write workload across the entire disk space.
Read Phase
Generates a random read workload across the disk space.
Read and Compare Phase
SLIDE 59
UPGRAID Joseph Naps, Ellen Wagner
Integrity Checker
Proceeds through three testing phases:
Write Phase
Generates a write workload across the entire disk space.
Read Phase
Generates a random read workload across the disk space.
Read and Compare Phase
Reads back in the original write workload and compares the data to ensure there was no data corruption.
SLIDE 60
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
SLIDE 61
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules
SLIDE 62
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
SLIDE 63
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential
SLIDE 64
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes
SLIDE 65
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate
SLIDE 66
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size
SLIDE 67
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size
SLIDE 68
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size duration of experiment
SLIDE 69
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size duration of experiment seed for the random number generator
SLIDE 70
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size duration of experiment seed for the random number generator
Output Variables
SLIDE 71
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size duration of experiment seed for the random number generator
Output Variables
actual duration of experiment
SLIDE 72
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size duration of experiment seed for the random number generator
Output Variables
actual duration of experiment average I/O time
SLIDE 73
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size duration of experiment seed for the random number generator
Output Variables
actual duration of experiment average I/O time standard deviation
SLIDE 74
UPGRAID Joseph Naps, Ellen Wagner
Workload Profiler
Generates a workload according to user specifications to test ABLE modules Input Variables
percent sequential fraction writes I/O request rate average I/O size maximum I/O size duration of experiment seed for the random number generator
Output Variables
actual duration of experiment average I/O time standard deviation throughput
SLIDE 75
UPGRAID Joseph Naps, Ellen Wagner
Future Work
SLIDE 76
UPGRAID Joseph Naps, Ellen Wagner
Future Work
Read Heuristic
SLIDE 77
UPGRAID Joseph Naps, Ellen Wagner
Future Work
Read Heuristic Testing and debugging of replication, indirection, and popularity code
SLIDE 78
UPGRAID Joseph Naps, Ellen Wagner
Future Work
Read Heuristic Testing and debugging of replication, indirection, and popularity code Reconstruction
SLIDE 79
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Read Heuristic
SLIDE 80
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Read Heuristic
A similar task is done in the RAID1 code.
SLIDE 81
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Read Heuristic
A similar task is done in the RAID1 code. We have looked into the code and think that it can be ported to UPGRAID with a few modifications.
SLIDE 82
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Testing and Debugging
SLIDE 83
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Testing and Debugging
Currently using autorwbench for the purpose of testing UPGRAID
SLIDE 84
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Testing and Debugging
Currently using autorwbench for the purpose of testing UPGRAID Once the system is more stable with autorwbench UPGRAID can be deployed on a file system.
SLIDE 85
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction
SLIDE 86
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction
Not considered in detail yet
SLIDE 87
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction
Not considered in detail yet Two main approaches exist
SLIDE 88
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction
Not considered in detail yet Two main approaches exist
Disk-Oriented Reconstruction (DOR)
SLIDE 89
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction
Not considered in detail yet Two main approaches exist
Disk-Oriented Reconstruction (DOR) Popularity-based Reconstruction (PRO)
SLIDE 90
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction
Not considered in detail yet Two main approaches exist
Disk-Oriented Reconstruction (DOR) Popularity-based Reconstruction (PRO)
An entirely new approach could be developed for UPGRAID
SLIDE 91
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via DOR
SLIDE 92
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via DOR
DOR works by generating a thread for each disk that is responsible for fulfilling requests to that disk for the purpose of rebuilding the data of the failed disk.
SLIDE 93
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via DOR
DOR works by generating a thread for each disk that is responsible for fulfilling requests to that disk for the purpose of rebuilding the data of the failed disk. There is also a master thread that is responsible for coordinating the actions of the disk threads.
SLIDE 94
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via DOR
DOR works by generating a thread for each disk that is responsible for fulfilling requests to that disk for the purpose of rebuilding the data of the failed disk. There is also a master thread that is responsible for coordinating the actions of the disk threads. It is possible that UPGRAID could work directly below the master thread and indirect rebuild requests for replicated blocks to the replicas stored on UPGRAID partitions.
SLIDE 95
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via PRO
SLIDE 96
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via PRO
PRO works by dividing the failed disk into “hot zones” and then rebuilding the zones based on the current access rate to that zone.
SLIDE 97
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via PRO
PRO works by dividing the failed disk into “hot zones” and then rebuilding the zones based on the current access rate to that zone. UPGRAID could sit above this process and use replicated stripes to improve this process.
SLIDE 98
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via PRO
PRO works by dividing the failed disk into “hot zones” and then rebuilding the zones based on the current access rate to that zone. UPGRAID could sit above this process and use replicated stripes to improve this process. This approach would likely be more complex but its popularity based operation seems like good fit with UPGRAID.
SLIDE 99
UPGRAID Joseph Naps, Ellen Wagner
Future Work - Reconstruction via PRO
PRO works by dividing the failed disk into “hot zones” and then rebuilding the zones based on the current access rate to that zone. UPGRAID could sit above this process and use replicated stripes to improve this process. This approach would likely be more complex but its popularity based operation seems like good fit with UPGRAID. It may be good if we defined these “hot zones” to align with the stripes of the RAID5 disk. This would make reconstruction using the replicated stripes easier.
SLIDE 100
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
SLIDE 101
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
Tested with autorwbench using one block (512 byte) I/O
- perations.
SLIDE 102
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
Tested with autorwbench using one block (512 byte) I/O
- perations.
10MB write workload
SLIDE 103
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
Tested with autorwbench using one block (512 byte) I/O
- perations.
10MB write workload 100MB read workload
SLIDE 104
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
Tested with autorwbench using one block (512 byte) I/O
- perations.
10MB write workload 100MB read workload Run in a virtual machine
SLIDE 105
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
SLIDE 106
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
RAID5 Average - 12.187 seconds
SLIDE 107
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
RAID5 Average - 12.187 seconds UPGRAID Average - 13.1984 seconds
SLIDE 108
UPGRAID Joseph Naps, Ellen Wagner
Extremely Preliminary Results
Due to the current instability of the system this data should be taken with a grain of salt.
SLIDE 109
UPGRAID Joseph Naps, Ellen Wagner