Performance of RAID5 disk arrays with read and write caching
- Jai Menon
- … show all 1 hide
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
In this paper, we develop analytical models and evaluate the performance of RAID5 disk arrays in normal mode (all disks operational), in degraded mode (one disk broken, rebuild not started) and in rebuild mode (one disk broken, rebuild started but not finished). Models for estimating rebuild time under the assumption that user requests get priority over rebuild activity have also been developed. Separate models were developed for cached and uncached disk controllers. Particular emphasis is on the performance of cached arrays, where the caches are built of Non-Volatile memory and support write caching in addition to read caching. Using these models, we evaluate the performance of arrayed and unarrayed disk subsystems when driven by a database workload such as those seen on systems running any of several popular database managers. In particular, we assume single-block accesses, flat device skew and little seek affinity.
With the above assumptions, we find six significant results. First, in normal mode, we find there is no difference in performance between subsystems built out of either small arrays or large arrays as long as the total number of disks used is the same. Second, we find that if our goal is to minimize the average response time of a subsystem in degraded and rebuild modes, it is better to use small arrays rather than large arrays in the subsystem. Third, we find the counter-intuitive result that if our goal is to minimize the average response time of requests to any one array in the subsystem, it is better to use large arrays than small arrays in the subsystem. We call this the best worst-case phenomenon.
Fourth, we find that when no caching is used in the disk controller, subsystems built out of arrays have a normal mode performance that is significantly worse than an equivalent unarrayed subsystem built of the same drives. For the specific drive, controller, workload and system parameters we used for our calculations, we find that, without a cache in the controller and operating at typical I/O rates, the normal mode response time of a subsystem built out of arrays is 50% higher than that of an unarrayed subsystem. In rebuild mode, we find that a subsystem built out of arrays can have anywhere from 100% to 200% higher average response time than an equivalent unarrayed subsystem.
Out fifth result is that, with cached controllers, the performance differences between arrayed and equivalent unarrayed subsystems shrink considerably. We find that the normal mode response time in a subsystem built out of arrays is only 4.1% higher than that of an equivalent unarrayed system. In degraded (rebuild) mode, a subsystem built out of small arrays has a response time 11% (13%) higher and a subsystem built out of large arrays has a response time 15% (19%) higher than an unarrayed subsystem.
Our sixth and last result is that cached arrays have significantly better response times and throughputs than equivalent uncached arrays. For one workload, a cached array with good hit ratios had 5 times the throughout and 10 to 40 times lower response times than the equivalent uncached array. With poor hit ratios, the cached array is still a factor of 2 better in throughput and a factor of 4 to 10 better in response time for this same workload.
We conclude that 3 design decisions are important when designing disk subsystems built out of RAID level 5 arrays. First, it is important that disk subsystems built out of arrays have disk controllers with caches, in particular Non-Volatile caches that cache writes in addition to reads. Second, if one were trying to minimize the worst response time seen by any user, one would choose disk array subsystems built out of large RAID level 5 arrays because of the best worst-case phenomenon. Third, if average subsystem response time is the most important design metric, the subsystem should be built out of small RAID level 5 arrays.
- ANSI Standard X3.131, Small Computers Systems Interface (SCSI),American National Standards Institute (ANSI) (New York, 1986).
- Bates, K. H., TeGrotenhuis, M. (1985) Shadowing Boosts System Reliability. Computer Design 24: pp. 129-137
- Chen, P. M., Gibson, G. and Patterson, D., An Evaluation of Redundant Arrays of Disks Using an Amdahl 5890,Proc. ACM SIGMETRICS Conf. on Measurement and Modeling of Computer Systems (Boulder, Colorado, May. 1990) pp. 75–85.
- Chen, P. M. and Patterson, D. A., Maximizing Performance in a Striped Disk Array,17th International Symposium on Computer Architecture (1990) pp. 22–331.
- Clark, B. E. et. al., Parity Spreading to Enhance Storage Access,United States Patent 4,761,785 (Aug. 1988).
- Conway, R. W., Maxwell, W. L. and Miller, L. W., Theory of Scheduling,Addison-Wesley (New York, 1967).
- Goldstein, S., Storage Performance—an Eight Year Outlook,IBM Santa Teresa Technical Report TR 03.308 (Santa Teresa, July 1987).
- Gray, J. N., Notes on Database Operating Systems,Operating Systems: An Advanced Course (New York, 1978).
- Gray, J. N., Horst, B. and Walker, M., Parity Striping of Disk Arrays: Low-Cost Reliable Storage With Acceptable Throughput,Tandem Computers Technical Report (Brisbane, Australia, August 1990) pp. 148–161.
- Holland, M. and Gibson, G. A., Parity Declustering for Continuous Operation in Disk Arrays,Proc. 5th ASPLOS (Boston, Oct. 1992) pp. 23–35.
- Hou, R. Y., Menon, J. and Patt, Y. N., Balancing IO Response Time and Rebuild Time in RAID5 arrays,Proc. Hawaii Int'l Conf. on System Sciences (Hawaii, Jan. 1993) pp. 70–79.
- Hyde, J., Personal Communication,IBM Publication Group (1990).
- IBM 3390 Direct Access Storage Introduction,IBM Publication GC26-4573 (San Jose, 1989).
- IMS-VS Version 1: Program Logic Manual,IBM Publication LY20-8004 (San Jose, 1977).
- Kleinrock, L., Queueing Systems, Vol.I,John Wiley (New York, 1975).
- Lawlor, F. D. (1981) Efficient Mass Storage Parity Recovery Mechanism. IBM Technical Disclosure Bulletin 24: pp. 986-987
- Lynch, W. C., Do Disk Arms Move?Performance Evaluation Review I 3,16 (July 1972).
- McNutt, Bruce, A Simple Statistical Model of Cache Reference Locality,CMG Proceedings (1991).
- Menon, J. M. and Hartung, M., The IBM 3990 Disk Cache,Compcon 1988 (San Francisco, June 1988).
- Menon, J. and Mattson, R. L., Performance of Arrays in Transaction Processing Environments,Proc. of 12th International Conference on Distributed Computing Systems (Yokohama, Japan, June 1992) pp. 302–309.
- Menon, J. and Cortney, J., The Architecture of a Fault-Tolerant Cached RAID Controller,20th Annual International Symposium on Computer Architecture (San Diego, California, May 1993) pp. 76–86.
- Muntz, R. R. and Lui, C. S., Performance Analysis of Disk Arrays Under Failure,16th VLDB Conference (Brisbane, Australia, 1990) pp. 162–173.
- Nelson, R., Tantawi, N. (1988) Approximate Analysis of Fork-Join Synchronization in Parallel Queues. IEEE Transactions on Computers 37: pp. 739-743
- Patterson, D., Gibson, G. and Katz, R. H., Reliable Arrays of Inexpensive Disks (RAID),ACM SIGMOD Conference 1988 (Chicago, June 1988) pp. 109–116.
- Teng, J., DB2 Buffer Pool Management,GUIDE 79 Conference (San, Jose, March 1991).
- Performance of RAID5 disk arrays with read and write caching
Distributed and Parallel Databases
Volume 2, Issue 3 , pp 261-293
- Cover Date
- Print ISSN
- Online ISSN
- Kluwer Academic Publishers
- Additional Links
- disk arrays
- non-volatile caches
- write caches
- storage subsystem performance
- Industry Sectors
- Jai Menon (1)
- Author Affiliations
- 1. IBM Almaden Research Center, 650 Harry Road, 95120-6099, San Jose, CA