Abstract
Non-structured and semi-structured large-scale data storage has become a hot research topic. This paper has, through analysis and research on the fast I/O mechanism of large-scale block data, designed a fast data access mechanism. The system can achieve efficient storage of large-scale small files, reduce the I/O frequency of metadata control information, improve the storage and access performance of small files, reduce the memory utilization of metadata and maximize the number of files supported by the system; meanwhile, the system can enhance the storage performance of large files by providing such files with high-speed parallel access interfaces.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Foster, I., Yong, Z., Raicu, I., et al.: Cloud computing and grid computing 360-degree compared, pp. 1–10 (2008)
Mell, P., Grance, T.: The NIST definition of cloud computing. National Institute of Standards and Technology (2011)
Li, G.J.: Scientific value of the study on big data. Communications of the CCFÂ 8 (2012)
Kuang, S.H., Li, B.: System architecture and application cases of cloud computing. Computer & Digital Engineering 245 (2010)
Wu, Y.W., Huang, X.M.: Cloud storage. Communications of the CCFÂ 5 (2009)
Weil, S.A., Pollack, K.T., Brandt, S.A., Miller, A.E.L.: Dynamic metadata management for petabyte-scale file systems. In: Proc. of the 2004 ACM/IEEE Conf. on Supercomputing (SC 2004). ACM Press, Pittsburgh (2004), doi:10.1109/SC.2004.22
Hua, Y., Jiang, H., Zhu, Y.F., Feng, D., Tian, L.: SmartStore: A new metadata organization paradigm with semantic-awareness fornext-generation file systems. In: Proc. of the FAST 2009, pp. 1–12. ACM Press, Portland (2009), doi:10.1145/1654059.1654070
Decandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s highly available key-value store. In: Proc. of the SOSP 2007, pp. 205–220. ACM Press, Stevenson (2007), doi:10.1145/1294261.1294281
Lakshman, A., Malik, P.: Cassandra: A decentralized structured storage system. ACM SIGOPS Operating Systems Review 44(2), 35–40 (2010), doi:10.1145/1773912.1773922
Ghemawat, S., Gobioff, H., Leung, S.T.: The Google file system. In: Proc. of the Symp. on Operating Systems Principles (SOSP 2003), pp. 29–43. ACM Press, Bolton (2003), doi:10.1145/945445.945450
Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: A distributed storage system for structured data. In: Proc. of the OSDI 2006, pp. 205–218. USENIX Association, Seattle (2006), http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/zh-CN//archive/bigtable-osdi06.pdf
The Apache Software Foundation. HDFS Architecture Guide (2009), http://hadoop.apache.org/common/docs/current/hdfs_design.html
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoop distributed file system. In: Proc. of the IEEE 26th Symp. on Mass Storage Systems and Technologies (MSST), pp. 1–10. IEEE, Lake Tahoe (2010), doi:10.1109/MSST.2010.5496972
Deng, Q.N., Chen, Q.: Cloud computing and key technologies. Development & Application of High Performance Computing 26 (2009)
Wang, Y.J., Sun, W.D., Zhou, S., Pei, X.Q., Li, X.Y.: Key technologies for distributed storage in the cloud computing environment. Journal of Software, Institute of Software Chinese Academy of Sciences 23(4), 962–986 (2012)
Chen, K., Zheng, W.M.: Cloud computing: system cases and research status. Journal of Software 20 (2009)
Luo, J.Z., Jin, J.H., Song, A.B., Dong, F.: Cloud computing: system architecture and key technologies. Journal on Communication 32 (2011)
Sun, J., Jia, X.Q.: Technical architecture of Google cloud computing platform and factors impacting its cost. Telecommunications Science (2010)
Zheng, L., Zhu, L.G., Zhao, Y.T., Yi, Q., Yan, C.P., Hu, H.Z., Yang, F.: Design of evaluation method of storage virtualization. Computer Engineering and Applications 46(36) (2010)
Zawodny, J.: Redis: Lightweight key/value Store That Goes the Extra Mile. Linux Magazine (August 31, 2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ying, W., Dongri, Y., Peng, L. (2014). CloStor: A Cloud Storage System for Fast Large-Scale Data I/O. In: Jeong, H., S. Obaidat, M., Yen, N., Park, J. (eds) Advances in Computer Science and its Applications. Lecture Notes in Electrical Engineering, vol 279. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41674-3_142
Download citation
DOI: https://doi.org/10.1007/978-3-642-41674-3_142
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41673-6
Online ISBN: 978-3-642-41674-3
eBook Packages: EngineeringEngineering (R0)