Abstract
To develop optimal parallel I/O subsystems, one must have a thorough understanding of the workload characteristics of parallel I/O and its exploitation of the associated parallel file system. Presented are the results of a study conducted to analyze the parallel I/O workloads of several applications on a parallel processor using the Vesta parallel file system. Traces of the applications are obtained to collect system events, communication events, and parallel I/O events. The traces are then analyzed to determine workload characteristics. The results show I/O request rates on the order of hundreds of requests per second, a large majority of requests are for small amounts of data (less than 1500 bytes), a few requests are for large amounts of data (on the order of megabytes), significant file sharing among processes within a job, and strong temporal, traditional spatial, and interprocess spatial locality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M.G. J.H. Hartman M.D. Kupfer K.W. Shirriff Baker and J.K. Ousterhout. Measurements of a distributed file system. ACM Symposium on Operating System Principles, pages 198–212., 1991.
S.J. C. Benveniste Baylor and Y. Hsu. Performance evaluation of a parallel i/o architecture. International Conference on Supercomputing, pages 404–413., July 1995.
M.L. A. Greenberg C. Stanfill Best and L.W. Tucker. Cmmd i/o: A parallel unix i/o. International Parallel Processing Symposium, pages 489–495., April 1993.
P.F. Corbett and D.G. Feitelson. Design and implementation of the vesta parallel file system. Scalable High Performance Computing Conference, pages 63–70., 1994.
P.F. D.G. Feitelson J-P. Prost Corbett and SJ. Baylor. Parallel access to files in the vesta file system. Supercomputing’93, pages 472–481., November 1993.
T.W. Crockett. File concepts for parallel i/o. Supercomputing’89, pages 574–579., 1989.
E. DeBenedictis and J.M. Rosario, ncube parallel i/o software. International Phoenix Conference on Computers and Communications, pages 117–124., April 1992.
J.M. del Rosario and A. Choudary. High performance i/o for parallel computers: Problems and prospects. Computer, 27(3):59–68., March 1994.
P.C. Dibble. A Parallel Interleaved File System. Univeristy of Rochester, March 1990.
Alverson et. al. The tera computer system. International Conference on Super-computing, pages 1–6., June 1990.
Corbett et. al. Parallel file systems for the ibm sp computers. IBM System Journal, 34(2):222–248.,1995.
Hayes et. al. A microporcessor-based hypercube supercomputer. IEEE Micro, 6(5):6–17., October 1986.
Leiserson et. al. The network architecture of the connection machine cm-5. 4th Symposium on Parallel Algorithms and Architectures, pages 272–285., June 1992.
International Business Machines Corporation. Scalable POWERparallel Systems High-Performance Technical Computing Solutions, March 1994.
E.T. Kalns and Y. Hsu. Video-on-demand using the vesta parallel file system. Third Annual Workshop on I/O in Parallel and Distributed Systems, pages 30–46., April 1995.
Kendal Square Research. KSR1 Technology Background, January 1992.
D. Kotz. Multiprocessor file system interfaces. International Conference on Parallel and Distributed Information Systems, pages 194–201., 1993.
D. Kotz and N. Nieuwejaar. Dynamic file-access characteristics of a production parallel scientific workload. Supercomputing’94, pages 640–649., November 1994.
O. Krieger and M. Stumm. Hfs: A flexible file system for large-scale multiprocessors. DAGS/PC Symposium, pages 6–14., June 1993.
P. Messina. The concurrent supercomputing consortium: Year 1. IEEE Parallel and Distributed Technology, 1(1):9–16., February 1993.
E.L. Miller and R.H. Katz. Input/output behavior of supercomputer applications. Supercomputing’91, pages 567–576., November 1991.
E.L. Miller and R.H. Katz. Rama: A file system for massively-parallel computers. IEEE Symposium on Mass Storage Systems, pages 163–168., April 1993.
ParaSoft Corporation. Express C Reference Guide, 1990.
B.K. Pasquale and G.C. Polyzos. A static analysis of i/o characteristics of scientific applications in a production workload. Supercomputing’93, pages 388–397., November 1993.
B.K. Pasquale and G.C. Polyzos. Dynamic i/o characterization of i/o intensive scientific applications. Supercomputing’94, pages 660–669., November 1994.
P. Pierce. A concurrent file system for a high parallel mass storage subsystem. Fourth Conference on Hypercube Computers and Applications, pages 155–160., 1989.
A.L.N. Reddy and P. Banerjee. A study of i/o behavior of perfect benchmarks on a multiprocessor. International Symposium on Computer Architecture, pages 312–321., 1990.
C.B. D.G. Shea D.G. Grice P.H. Hochschild Stunkel and M. Tsao. The sp1 high-performance switch. Scalable High Performance Computing Conference, May 1994.
C.E. Y.-H. Liu C. Benveniste L. Chen Wu and W.H. Chiang. Trace-based analysis and tuning for distributed parallel applications. International Conference on Parallel and Distributed Systems, pages 716–723., December 1994.
C.E. Y.-H. Liu Y. Hsu Wu. Timestamp consistency and trace-driven analysis for distributed parallel systems. International Parallel Processing Symposium, pages 680–686., April 1995.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Kluwer Academic Publishers
About this chapter
Cite this chapter
Baylor, S.J., Wu, C.E. (1996). Parallel I/O Workload Characteristics Using Vesta. In: Jain, R., Werth, J., Browne, J.C. (eds) Input/Output in Parallel and Distributed Computer Systems. The Kluwer International Series in Engineering and Computer Science, vol 362. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-1401-1_7
Download citation
DOI: https://doi.org/10.1007/978-1-4613-1401-1_7
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4612-8607-3
Online ISBN: 978-1-4613-1401-1
eBook Packages: Springer Book Archive