HPCC 2007: High Performance Computing and Communications pp 7-18 | Cite as
A Windows-Based Parallel File System
Abstract
Parallel file systems are widely used in clusters to provide high performance I/O. However, most of the existing parallel file systems are based on UNIX-like operating systems. We use the Microsoft .NET framework to implement a parallel file system for Windows. We also implement a file system driver to support existing applications written with Win32 APIs. In addition, a preliminary MPI-IO library is also developed. Applications using MPI-IO could achieve the best performance using our parallel file system, while the existing binaries could benefit from the system driver without any modifications. In this paper, the design and implementation of our system are described. File system performance using our preliminary MPI-IO library and system driver is also evaluated. The results show that the performance is scalable and limited by the network bandwidth.
Preview
Unable to display preview. Download preview PDF.
References
- 1.Adiga, N.R., Blumrich, M., Liebsch, T.: An overview of the BlueGene/L supercomputer. In: Proceedings of the 2002 ACM/IEEE Conference on Supercomputing, Baltimore, Maryland, pp. 1–22 (2002)Google Scholar
- 2.Myricom: Myrinet, http://www.myri.com/
- 3.InfiniBand Trade Association: Infiniband, http://www.infinibandta.org/
- 4.Pérez, J.M., Carretero, J., García, J.D.: A Parallel File System for Networks of Windows Worstations. In: ACM International Conference on Supercomputing (2004)Google Scholar
- 5.Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: A parallel file system for linux clusters. In: 4th Annual Linux Showcase and Conference, Atlanta, GA, pp. 317–327 (2000)Google Scholar
- 6.Ligon III, W. B., Ross, R.B.: An Overview of the Parallel Virtual File System. In: 1999 Extreme Linux Workshop (1999) Google Scholar
- 7.Kleiman, S., Walsh, D., Sandberg, R., Goldberg, D., Lyon, B.: Design and implementation of the sun network filesystem. In: Proc. Summer USENIX Technical Conf., pp. 119–130 (1985)Google Scholar
- 8.Hertel, C.R.: Implementing CIFS: The Common Internet File System. Prentice-Hall, Englewood Cliffs (2003)Google Scholar
- 9.Russinovich, M.E., Solomon, D.A.: Microsoft Windows Internals, Microsoft Windows Server 2003, Windows XP, and Windows 2000, 4th edn. Microsoft Press, Redmond (2004)Google Scholar
- 10.Corbett, P., Feitelson, D., Fineberg, S., Hsu, Y., Netzberg, W., Prost, J., Snir, M., Traverset, W., Wong, P.: 32. In: Overview of the MPI-IO Parallel IO Interface, IEEE and Wiely Interscience, Los Alamitos (2002)Google Scholar
- 11.Ligon III, W.B., Ross, R.B.: Implementation and performance of a parallel file system for high performance distributed applications. In: Proceedings of the Fifth IEEE International Symposium on High Performance Distributed Computing, pp. 471–480. IEEE Computer Society Press, Los Alamitos (1996)CrossRefGoogle Scholar
- 12.Feitelson, D.G., Corbett, P.F., Prost, J.-P.: Performance of the vesta parallel file system. In: 9th Internationl Parallel Processing Symposium, pp. 150–158 (1995)Google Scholar