Advertisement

Embedding an Extra Layer of Data Compression Scheme for Efficient Management of Big-Data

  • Sayan Pal
  • Indranil Das
  • Suvajit Majumder
  • Amit Kr. Gupta
  • Indrajit Bhattacharya
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 340)

Abstract

Use of Smartphone as mobile nodes in different communication infrastructures is excessively explored in recent years. Such smart phones can be considered as a good candidate for situations like Disaster Management, where there is no infrastructure available to support communication and connectivity among the group members is a prime objective. Disaster rescue operations are generally based on location intensive operations including neighboring nodes’ locations and their availability. The storage limitations of such devices ask for suitable strategies to store information efficiently. In this work, a method has been proposed that employs an extra layer of compression, while storing location data in the form of latitude-longitude (lat-long) pairs, to the HBase database. Location data in a mobile network is big-data, as continuous collection of such information adds numerous data inputs. By incurring a negligible overhead on the system in the form of small encoding and decoding time, the proposed method obtains almost 70 % compression ratio, even for thousands of input data. In this work Huffman lossless encoding scheme has been used.

Keywords

Disaster management Location data Hbase Compression Huffman code 

References

  1. 1.
    Fall, K.: A delay-tolerant network architecture for challenged internets. In: Proceedings of the 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM ‘03), pp. 27–34. ACM, New York (2003)Google Scholar
  2. 2.
    Japan Earthquake Tsunami 2011: http://en.wikipedia.org/wiki/2011_T%C5%8Dhoku-earthquake_and_tsunami. Accessed 15 Sept 2014
  3. 3.
  4. 4.
    Big Database Documentation: http://hbase.apache.org/. Accessed 15 Sept 2014
  5. 5.
    GeoHash Tips: http://geohash.org/site/tips.html. Accessed 15 Sept 2014
  6. 6.
    Google Maps API: https://developers.google.com/maps/. Accessed 15 Sept 2014
  7. 7.
    Big Data Access Language—Pig Latin: http://pig.apache.org/docs/r0.12.0/basic.html. Accessed 15 Sept 2014
  8. 8.
    Spyder IDE for python: https://code.google.com/p/spyderlib/. Accessed 15 Sept 2014
  9. 9.
    Pentaho Data Integration Software: http://www.pentaho.com/product/data-integration. Accessed 15 Sept 2014
  10. 10.
    McGrath, S.: Artemis: A vision for remote triage and emergency management information integration. Dartmouth University (2003)Google Scholar
  11. 11.
    Martín-Campillo, A., Martí, R., Yoneki, E., Crowcroft, J.: Electronic triage tag and opportunistic networks in disasters. In: Proceedings of the Special Workshop on Internet and Disasters (SWID ‘11). ACM, New York, Article 6, 10 p. (2011)Google Scholar
  12. 12.
    Malan, D., Fulford-Jones, T.R.F., Welsh, M., Moulton, S.: CodeBlue: an ad hoc sensor network infrastructure for emergency medical care. In: Proceedings of the MobiSys 2004 Workshop on Applications of Mobile Embedded Systems (WAMES 2004), pp. 12–14. Boston, MA (2004)Google Scholar
  13. 13.
    Kopetz, H.: Internet of things. In: Real-Time Systems, pp. 307–323. Springer, New York (2011)Google Scholar
  14. 14.
    Chen, M., Mao, S., Liu, Y.: Big data storage. Big Data. pp. 33–49. Springer, Berlin (2014)Google Scholar
  15. 15.
    Huffman, D.A.: A method for the construction of minimum redundancy codes. Proc. IRE 40(9), 1098–1101 (1952)CrossRefMATHGoogle Scholar
  16. 16.
    Manber, U.: A text compression scheme that allows fast searching directly in the compressed file. ACM Trans. Inf. Syst. (TOIS) 15(2), 124–136 (1997)CrossRefGoogle Scholar
  17. 17.
    Moffat, A., Zobel, J.: Coding for compression in full-text retrieval systems. In: Proceedings IEEE Data Compression Conference, pp. 72–81 (1992)Google Scholar
  18. 18.
    Gopal, B., Manber, U.: A fixed-dictionary approach to fast searching in compressed files. (Manuscript)Google Scholar
  19. 19.
    Klein, S.T., Bookstein, A., Deerwester, S.: Storing text retrieval systems on CD-ROM: compression and encryption considerations. ACM Trans. Inf. Syst. 7, 230–245 (1989)CrossRefGoogle Scholar

Copyright information

© Springer India 2015

Authors and Affiliations

  • Sayan Pal
    • 1
  • Indranil Das
    • 1
  • Suvajit Majumder
    • 1
  • Amit Kr. Gupta
    • 2
  • Indrajit Bhattacharya
    • 2
  1. 1.West Bengal University of TechnologySalt Lake, KolkataIndia
  2. 2.Kalyani Government Engineering CollegeKalyaniIndia

Personalised recommendations