Distributed and Parallel Databases

, Volume 1, Issue 1, pp 33–51 | Cite as

Practical prefetching techniques for multiprocessor file systems

  • David Kotz
  • Carla Schlatter Ellis


Improvements in the processing speed of multiprocessors are outpacing improvements in the speed of disk hardware. Parallel disk I/O subsystems have been proposed as one way to close the gap between processor and disk speeds. In a previous paper we showed that prefetching and caching have thepotential to deliver the performance benefits of parallel file systems to parallel applications. In this paper we describe experiments withpractical prefetching policies that base decisions only on on-line reference history, and that can be implemented efficiently. We also test the ability of those policies across a range of architectural parameters.


multiprocessor file systems parallel I/O file caching prefetching 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    M.G. Baker, J.H. Hartman, M.D. Kupfer, K.W. Shirriff, and J.K. Ousterhout, “Measurements of a distributed file system,” inProc. 13th ACM Symp. Operating Systems Principles, 1991, pp. 198–212.Google Scholar
  2. 2.
    BBN Advanced Computers.Butterfly Products Overview, 1987.Google Scholar
  3. 3.
    J.L. Bell, “A specialized data management system for parallel execution of particle physics codes,” inACM SIGMOD Conf., 1988, pp. 277–285.Google Scholar
  4. 4.
    L.-F. Cabrera and D.D.E. Long, “Swift: Using distributed disk striping to provide high I/O data rates,” Technical Report CRL-91-46, UC Santa Cruz, 1991,Computing Systems. To appear.Google Scholar
  5. 5.
    T.W. Crockett, “File concepts for parallel I/O,” inProc. Supercomputing '89, 1989, pp. 574–579.Google Scholar
  6. 6.
    E. DeBenedictus and J.M. del Rosario, “nCUBE parallel I/O software,” inEleventh Ann. IEEE Int. Phoenix Conf. Computers Communications (IPCCC), April 1992, pp. 0117–0124.Google Scholar
  7. 7.
    P.C. Dibble, “A parallel interleaved file system,” Ph.D. thesis, University of Rochester, 1990.Google Scholar
  8. 8.
    P. Dibble, M. Scott, and C. Ellis, “Bridge: A high-performance file system for parallel processors,” inProc. 8th Int. Conf. Distributed Computer Systems, 1988, pp. 154–161.Google Scholar
  9. 9.
    R. Floyd, “Short-term file reference patterns in a UNIX environment,” Technical Report 177, Dept. of Computer Science, Univ. of Rochester, 1986.Google Scholar
  10. 10.
    R.A. Floyd, and C.S. Ellis, “Directory reference patterns in hierarchical file systems,”IEEE Trans. Knowledge Data Eng., vol. 1, pp. 238–247, 1989.Google Scholar
  11. 11.
    J.C. French, T.W. Pratt, and M. Das, “Performance measurement of a parallel input/output system for the Intel iPSC/2 hypercube,” inProc. 1991 ACM Sigmetrics Conf. Measurement and Modeling Computer Systems, 1991, pp. 178–187.Google Scholar
  12. 12.
    “Intel beefs up its iPSC/2 supercomputer's I/O and memory capabilities,”Electronics, p. 24, Nov. 1988.Google Scholar
  13. 13.
    iPSC/2 I/O facilities, Intel Corporation, 1988. Order number 280120-001.Google Scholar
  14. 14.
    Paragon XP/S product overview, Intel Corporation, 1991.Google Scholar
  15. 15.
    M.Y. Kim, “Synchronized disk interleaving,”IEEE Trans. Comput., vol. C-35, pp. 978–988, 1986.Google Scholar
  16. 16.
    D. Kotz, “Prefetching and caching techniques in file systems for MIMD multiprocessors,” Ph.D. thesis, Duke University, 1991. Available as technical report CS-1991-016.Google Scholar
  17. 17.
    D. Kotz and C.S. Ellis, “Prefetching in file systems for MIMD multiprocessors,”IEEE Trans. Parallel Distributed Syst., vol. 1, pp. 218–230, 1990.Google Scholar
  18. 18.
    D. Kotz and C.S. Ellis, “Caching and writeback policies in parallel file systems,” in1991 IEEE Symposium Parallel and Distributed Processing, 1991, pp. 60–67, to appear inParallel Distributed Comput.Google Scholar
  19. 19.
    E.L. Miller and R.H. Katz, “Input/output behavior of supercomputer applications,” inProc. Supercomputing '91, 1991, pp. 567–576.Google Scholar
  20. 20.
    nCUBE Corporation, “nCUBE 2 supercomputers: Technical overview,” nCUBE brochure, 1990.Google Scholar
  21. 21.
    J. Ousterhout, H.D. Costa, D. Harrison, J. Kunze, M. Kupfer, and J. Thompson, “A trace driven analysis of the UNIX 4.2 BSD file system,” inProc. 10th ACM Symp. Operating Systems Principles, 1985, pp. 15–24.Google Scholar
  22. 22.
    J. Ousterhout and F. Douglis, “Beating the I/O bottleneck: A case for log-structured file systems,”ACM Oper. Syst. Rev., vol. 23, pp. 11–28, Jan. 1989.Google Scholar
  23. 23.
    D. Patterson, G. Gibson, and R. Katz, “A case for redundant arrays of inexpensive disks (RAID),” inACM SIGMOD Conf., 1989, pp. 109–116.Google Scholar
  24. 24.
    P. Pierce, “A concurrent file system for a highly parallel mass storage system,” in4th Conf. Hypercube Concurrent Computers and Applications, 1989, pp. 155–160.Google Scholar
  25. 25.
    T.W. Pratt, J.C. French, P.M. Dickens, and S.A. Janet, Jr., “A comparison of the architecture and performance of two parallel file systems,” in4th Conf. Hypercube Concurrent Computers and Applications, 1989, pp. 161–166.Google Scholar
  26. 26.
    K. Salem and H. Garcia-Molina, “Disk striping,” inIEEE 1986 Conf. Data Engineering, 1986, pp. 336–342.Google Scholar
  27. 27.
    A.J. Smith, “Sequential program prefetching in memory hierarchies,”IEEE Comput., vol. 11, pp. 7–21, Dec. 1978.Google Scholar
  28. 28.
    A.J. Smith, “Sequentiality and prefetching in database systems,”ACM Trans. Database Syst. vol. 3, pp. 223–247, 1978.Google Scholar
  29. 29.
    A.J. Smith, “Cache memories,”Comput. Surv. vol. 14, pp. 473–530, 1982.Google Scholar
  30. 30.
    A.J. Smith, “Disk cache-miss ratio analysis and design considerations,”ACM Trans. Comput. Syst. vol. 3, pp. 161–203. 1985.Google Scholar
  31. 31.
    A. Tucker and A. Gupta, “Process control and scheduling issues for multi-programmed sharedmemory multiprocessors,” inProc. 12th ACM Symp. Operating Systems Principles, 1989, pp. 159–166.Google Scholar

Copyright information

© Kluwer Academic Publishers 1993

Authors and Affiliations

  • David Kotz
    • 1
  • Carla Schlatter Ellis
    • 2
  1. 1.Department of Mathematics and Computer ScienceDartmouth CollegeHanover
  2. 2.Department of Computer ScienceDuke UniversityDurham

Personalised recommendations