Advertisement

A Metadata Management Strategy Based on Event-Classification in Intelligent Transportation System

  • Yayun Su
  • Yaying ZhangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9528)

Abstract

With the explosive growth of data information, the object-oriented storage system has been widely used. This paper proposed a metadata management strategy based on Distributed File System-Ceph in terms of event classification, taking advantage of the characteristics of data in urban traffic system. The large amount of data with a wide variety of sources and data types was first classified by machine learning, and a classification model was established. Then, improvements on load balancing were made to the existing Ceph Load Balancing Strategy of metadata partition. The metadata partitioning is to assign and migrate metadata obtained from the event classification model to the target server chosen by the fuzzy optimum method. Experimental results show that the proposed load balancing strategy based on event classification can not only make the overall load of the metadata servers in a relatively stable state but also make the migration times less than that of other algorithms. The extra overhead of the system is also reduced.

Keywords

Metadata management strategy Ceph Event classification Traffic data Load balancing 

Notes

Acknowledgments

This research was supported by the International Science & Technology Cooperation Program of China (2012DFG11580).

References

  1. 1.
    Dimitrakopoulos, G., Demestichas, P.: Intelligent transportation systems. IEEE Veh. Technol. Mag. 5(1), 77–84 (2010)CrossRefGoogle Scholar
  2. 2.
    Zhao, Z., Fang, J., Ding, W., et al.: An integrated processing platform for traffic sensor data and its applications in intelligent transportation systems. In: 2014 IEEE World Congress on Services (SERVICES), pp. 161–168. IEEE Computer Society (2014)Google Scholar
  3. 3.
    Li, W., Xue, W., Shu, J., et al.: Dynamic hashing: adaptive metadata management for petabyte-scale file system. In: proceeding of the 23st IEEE/14th NASA Goddard Conference on Mass Storage System and Technologies (2006)Google Scholar
  4. 4.
    Lin, Y., Li, R., Xu, Z., et al.: A dynamic method for metadata partitioning based on intensive access of spatial data. In: 6th IEEE Joint International Conference on Information Technology and Artificial Intelligence (ITAIC), pp. 177–180. IEEE (2011)Google Scholar
  5. 5.
    Weil, S.A., Brandt, S.A., Miller, E.L., et al.: Ceph: A scalable, high-performance distributed file system. In: Proceedings of the 7th symposium on Operating systems design and implementation. USENIX Association, pp. 307–320 (2006)Google Scholar
  6. 6.
    Outerhout, J.K., Da Costa, H., Harrison, D., et al.: A Trace-driven analysis of the Unix 4.2 BSD file system. ACM (1985)Google Scholar
  7. 7.
    Roselli, D.S., Lorch, J.R., Anderson, T.E.: A comparison of file system workloads. In: USENIX Annual Technical Conference, General Track, pp. 41–54 (2000)Google Scholar
  8. 8.
    Yan, J., Zhu, Y., Xiong, H., et al.: A design of metadata server cluster in large distributed object-based storage. In: MSST, pp. 199–205 (2004)Google Scholar
  9. 9.
    Qin, M., Wang, Y., Cui, Z., et al.: The design and realization of an advance urban traffic surveillance and Management system. In: The 6th World Congress on Intelligent Control and Automation (2006)Google Scholar
  10. 10.
    Brandt, S.A., Miller, E.L., Long, D.D.E., Xue, L.: Efficient metadata management in large distributed storage systems. In: Proceedings of the 20th IEEE/the 11th NASA Goddard Conference on Mass Storage Systems and Technologies, IEEE Computer Society, Washington (2003)Google Scholar
  11. 11.
    Honicky, R.J., Miller, E.L.: Replication under scalable hashing: a family of algorithms for scalable decentralized data distribution. In: International Symposium on Parallel and Distributed Processing, vol. 96a. IEEE Computer Society (2004)Google Scholar
  12. 12.
    Ceph: A Linux PB-level distributed file system. https://www.ibm.com/developerworks/cn/linux/l-ceph/
  13. 13.
    Weil, S.A., Brandt, S.A., Miller, E.L., et al.: Intelligent metadata management for a petabyte-scale file system. In: 2nd Intelligent Storage Workshop (2004)Google Scholar
  14. 14.
    Xin, Q., Miller, E.L., Schwarz, T., Long, D.D.E., et al.: Reliability mechanisms for very large storage systems. In: Proceedings of the 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST), pp. 146–156 (2003)Google Scholar
  15. 15.
    Hua, Y., Zhu, Y., Jiang, H., et al.: Supporting scalable and adaptive metadata management in ultralarge-scale file system. IEEE Trans. Parallel Distrib. Syst. 22(4), 580–593 (2011)CrossRefGoogle Scholar
  16. 16.
    Zhou, G., Lan, Q., Chen, J.: A dynamic metadata equipotent subtree partition policy for mass storage system. In: FCST, pp. 29–34. IEEE Computer Society (2007)Google Scholar
  17. 17.
    Chen, S.: Optimum selecting theory and model for fuzzy design. Syst. Eng. 8, 55–61 (1990)Google Scholar
  18. 18.
    Zhang, J., Qian, W., Xu, X., et al.: WLBS: a weight-based metadata server cluster load balancing strategy. Int. J. Adv. Comput. Technol. (IJACT) 14(9), 77–85 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Key Laboratory of Embedded System and Service ComputingTongji UniversityShanghaiChina

Personalised recommendations