Skip to main content
SpringerLink
Log in

Information Intelligent Management System Based on Hadoop

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

In order to explore the information management, the cloud computing technology is applied to the field of geographic information system, and remote sensing data storage and management system based on Hadoop is studied and realized. The main function of this system includes that the remote sensing data storage module provides the remote sensing data download function for data administrator, supports HTTP protocol and FTP protocol multi-threaded distributed HTTP download. The parallel constructing algorithm of remote sensing image of Pyramid based on Map Reduce is realized by the module, and layered cutting and block storage of massive remote sensing data are carried out. The GDAL open source library suitable for fast read raster data is used and it provides data resource for remote sensing data parallel cutting. In addition, the Geo Web Cache open source tile map service middleware is adopted and HBase is introduced as the storage support of tiles, which can deal with a large number of users’ visit, including loading and drag of maps. The system test is carried out to verify the effectiveness and practicability of the method proposed. The test results can show that the remote sensing data storage and management system based on Hadoop can effectively handle remote sensing data and improve the user’s experience. It is concluded that the information management system has high effectiveness and good practicability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Liroz-Gistau, M., Akbarinia, R., Agrawal, D., & Valduriez, P. (2016). Fp-Hadoop: Efficient processing of skewed mapreduce jobs. Information Systems, 60, 69–84.

    Article  Google Scholar 

  2. He, H., Du, Z., Zhang, W., & Chen, A. (2016). Optimization strategy of Hadoop small file storage for big data in healthcare. Journal of Supercomputing, 72(10), 1–12.

    Article  Google Scholar 

  3. Park, D., Wang, J., & Kee, Y. S. (2016). In-storage computing for Hadoop MapReduce framework: Challenges and possibilities. IEEE Transactions on Computers, PP(99), 1.

    Article  Google Scholar 

  4. Magana-Zook, S., Gaylord, J. M., Knapp, D. R., Dodge, D. A., & Ruppert, S. D. (2016). Large-scale seismic waveform quality metric calculation using Hadoop. Computers & Geosciences, 94, 18–30.

    Article  Google Scholar 

  5. Li, Z., & Shen, H. (2017). Measuring scale-up and scale-out Hadoop with remote and local file systems and selecting the best platform. IEEE Transactions on Parallel and Distributed Systems, PP(99), 3201–3214.

    Article  Google Scholar 

  6. Hodor, P., Chawla, A., Clark, A., & Neal, L. (2016). Cl-dash: Rapid configuration and deployment of Hadoop clusters for bioinformatics research in the cloud. Bioinformatics, 32(2), 301–303.

    Google Scholar 

  7. Um, J. H., Lee, S., Kim, T. H., Jeong, C. H., Song, S. K., & Jung, H. (2016). Distributed RDF store for efficient searching billions of triples based on Hadoop. Journal of Supercomputing, 72(5), 1825–1840.

    Article  Google Scholar 

  8. Li, C., Chen, T., He, Q., Zhu, Y., & Li, K. (2016). Mruninovo: An efficient tool for de novo peptide sequencing utilizing the Hadoop distributed computing framework. Bioinformatics, 33(6), 944.

    Google Scholar 

  9. Ferraro, P. U., Roscigno, G., Cattaneo, G., & Giancarlo, R. (2017). Fastdoop: A versatile and efficient library for the input of FASTA and FASTQ files for MapReduce Hadoop bioinformatics applications. Bioinformatics, 33(10), 1575.

    Google Scholar 

  10. Fu, X., Gao, Y., Luo, B., Du, X., & Guizani, M. (2017). Security threats to Hadoop: Data leakage attacks and investigation. IEEE Network, PP(99), 12–16.

    Google Scholar 

  11. Cai, X., Li, F., Li, P., Ju, L., & Jia, Z. (2017). SLA-aware energy-efficient scheduling scheme for Hadoop YARN. Journal of Supercomputing, 73(8), 3526–3546.

    Article  Google Scholar 

  12. Nguyen, M. C., Won, H., Son, S., Gil, M. S., & Moon, Y. S. (2017). Prefetching-based metadata management in advanced multitenant Hadoop. Journal of Supercomputing, 73(2), 1–21.

    Google Scholar 

Download references

Acknowledgements

The study was supported by “Science and Technology Plan of Ministry of Housing and Urban–Rural Development of China (Grant No. 2016-R4-014)”.

Author information

Authors and Affiliations

  1. College of Civil Science and Engineering, Yangzhou University, Yangzhou, China

    Zhenguo Zhou & Zhenggang Huo

Authors
  1. Zhenguo Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenggang Huo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenguo Zhou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Z., Huo, Z. Information Intelligent Management System Based on Hadoop. Wireless Pers Commun 102, 3803–3812 (2018). https://doi.org/10.1007/s11277-018-5411-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-018-5411-4

Keywords

Search

Navigation