Journal of Real-Time Image Processing

, Volume 12, Issue 2, pp 343–355 | Cite as

Efficient algorithm for transferring a real-time HEVC stream with haptic data through the internet

  • George Kokkonis
  • Kostas E. Psannis
  • Manos Roumeliotis
  • Yutaka Ishibashi
Special Issue Paper


It is widely accepted that the growth of Internet and the improvement of Internet’s network conditions helped real-time applications to flourish. The demand for Ultra-High Definition video is constantly increasing. Apart from video and sound, a new kind of real-time data is making its appearance, haptic data. The efficient synchronization of video, audio, and haptic data is a rather challenging effort. The new High-Efficiency Video Coding (HEVC) is quite promising for real-time ultra-high definition video transferring through the Internet. This paper presents related work on High-Efficiency Video Coding. It points out the challenges and the synchronization techniques that have been proposed for synchronizing video and haptic data. Comparative tests between H.264 and HEVC are undertaken. Measurements for the network conditions of the Internet are carried out. The equations for the transferring delay of all the inter-prediction configurations of the HEVC are defined. Finally, it proposes a new efficient algorithm for transferring a real-time HEVC stream with haptic data through the Internet.


Haptics HEVC High-Efficiency Video Coding Algorithm Synchronization techniques Inter-prediction Haptic data 


  1. 1.
    Ohm, J., Sullivan, G., Schwarz, H., Tan, T.K., Wiegand, T.: Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE. Trans. Circuits. Syst. Video. Technol. 22, 1669–1684 (2012)CrossRefGoogle Scholar
  2. 2.
    Hanhart, P., Rerabek, M., De Simone, F., Ebrahimi, T.: Subjective quality evaluation of the upcoming HEVC video compression standard. Proc. SPIE. 8499, 84990V1–84990V13 (2012)CrossRefGoogle Scholar
  3. 3.
    Garcia, R., Kalva, H.: Subjective evaluation of HEVC and avc/h.264 in mobile environments. IEEE Tran. Consum. Electron. 60, 116–123 (2014)CrossRefGoogle Scholar
  4. 4.
    Nightingale, J., Wang, Q., Grecos, C.: Benchmarking real-time HEVC streaming. Proc. SPIE. 8437, 843701D1–84370D14 (2012)Google Scholar
  5. 5.
    Panayides, A., Antoniou, Z., Pattichis, M., Pattichis, C.: The use of h.264/avc and the emerging high efficiency video coding (HEVC) standard for developing wireless ultrasound video telemedicine systems. In: Proc. Forty Sixth Asilomar Conf. on Signals, Systems and Computers (ASILOMAR), pp. 337–341 (2012)Google Scholar
  6. 6.
    Bossen, F., Bross, B., Suhring, K., Flynn, D.: HEVC complexity and implementation analysis. IEEE. Tran. Circuits. Syst. Video. Technol. 22(12), 1685–1696 (2012)CrossRefGoogle Scholar
  7. 7.
    Kim I-K., McCann K., Sugimoto K., Bross B., Woo-Jin H.:Hm9: High efficiency video coding (HEVC) test model 9 encoder description, in 9th JCT-VC Meeting, Switzerland, 2012, pp. 10–11Google Scholar
  8. 8.
    Sullivan, G., Ohm, J., Han, W.-J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE. Trans. Circuits. Syst. Video. Technol. 22, 1649–1668 (2012)CrossRefGoogle Scholar
  9. 9.
    Chi, C.C., Alvarez-Mesa, M., Juurlink, B., Clare, G., Henry, F., Pateux, S., Schierl, T.: Parallel scalability and efficiency of HEVC parallelization approaches. IEEE. Trans. Circuits. Syst. Video. Technol. 22(12), 1827–1838 (2012)CrossRefGoogle Scholar
  10. 10.
    Alvarez-Mesa M., Chi C., Juurlink B., George V., Schierl T.: Parallel video decoding in the emerging HEVC standard. In: IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), pp. 1545–1548 (2012)Google Scholar
  11. 11.
    Saddik, E.: The potential of haptics technologies. IEEE. Instrum. Meas. Mag. 10, 10–17 (2007)CrossRefGoogle Scholar
  12. 12.
    Eid, M., Cha, J., El Saddik, A.: Admux: an adaptive multiplexer for haptic-audio-visual data communication. IEEE. Tran. Instrum Meas. 60, 21–31 (2011)CrossRefGoogle Scholar
  13. 13.
    Iwata, K., Ishibashi, Y., Fukushima, N., Sugawara, S.: QoE assessment in haptic media, sound, and video transmission: effect of playout buffering control. Comput. Entertain. 8, 12:1–12:14 (2010)CrossRefGoogle Scholar
  14. 14.
    Suzuki, N., Katsura S.: Evaluation of QoS in haptic communication based on bilateral control. In IEEE Int. Conf. on Mechatronics (ICM), pp. 886–891 (2013)Google Scholar
  15. 15.
    Isomura E., Tasaka S., Nunome, T.: A multidimensional qoe monitoring system for audiovisual and haptic interactive IP communications. In IEEE Consumer Communications and Networking Conference (CCNC), pp. 196–202 (2013)Google Scholar
  16. 16.
    Hamam, A., El Saddik, A.: Toward a mathematical model for quality of experience evaluation of haptic applications. IEEE. Tran. Instrum. Meas. 62, 3315–3322 (2013)CrossRefGoogle Scholar
  17. 17.
    Morimitsu, H., Katsura, S., Tomizuka, M.: Design of force compensator with variable gain for bilateral control system under time delay. In IEEE Int. Symposium on Ind. Electron. (ISIE), pp. 1–6 (2013)Google Scholar
  18. 18.
    Alturki, A., Alnuem, M.: Study of the effects of using some QoS mechanisms on haptic transmission—using opnet modeler. In IEEE Int. Workshop on Haptic Audio Visual Environments and Games (HAVE), pp. 94–101 (2011)Google Scholar
  19. 19.
    Ishibashi, Y., Tasaka, S., Tachibana, Y.: Adaptive causality and media synchronization control for networked multimedia applications. In: IEEE Int. Conf. on Communications, vol. 3, pp. 952–958 (2001)Google Scholar
  20. 20.
    Bartl, A., Diaz-Cacho, M., Barreiro, A., Delgado, E.: Passivity framework and traffic reduction for the teleoperation of a gantry crane. In: 39th Ann. Conf. IEEE Ind. Electron. Society (IECON), pp. 3675–3680 (013)Google Scholar
  21. 21.
    Sakaino, S., Sato, T., Ohnishi, K.: Precise position/force hybrid control with modal mass decoupling and bilateral communication between different structures. IEEE. Tran. Ind. Inf. 7, 266–276 (2011)CrossRefGoogle Scholar
  22. 22.
    Zeng, Q., Ishibashi, Y., Fukushima, N., Sugawara, S., Psannis, K.: Influences of inter-stream synchronization errors among haptic media, sound, and video on quality of experience in networked ensemble. In: IEEE 2nd Global Conf. on Consumer Electronics (GCCE), pp. 466–470 (2013)Google Scholar
  23. 23.
    Schierl, T., Hannuksela, M., Wang, Y.-K., Wenger, S.: System layer integration of high efficiency video coding. IEEE. Tran. Circuits Syst. Video. Technol. 22, 1871–1884 (2012)CrossRefGoogle Scholar
  24. 24.
    Ishibashi, Y., Tasaka, S.: A comparative survey of synchronization algorithms for continuous media in network environments. In: Proc. 25th Ann. IEEE Conf. on Local Computer Networks, pp. 337–348 (2000)Google Scholar
  25. 25.
    Ishibashi, Y., Tasaka, S., Ogawa, H.:A comparison of media synchronization quality among reactive control schemes. In: IEEE Proc. 14th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), vol. 1, pp. 77–84 (2001)Google Scholar
  26. 26.
    Ishibashi Y., Tasaka, S., Tsuji, A.: Measured performance of a live media synchronization mechanism in an atm network. In: IEEE Int. Conf. on Communications, vol. 3, pp. 1348–1354 (1996)Google Scholar
  27. 27.
    Ishibashi, Y., Kanbara, T., Tasaka, S.: Inter-stream synchronization between haptic media and voice in collaborative virtual environments. In Proc. 12th Ann. ACM Int.l Conf. on Multimedia, New York, pp. 604–611 (2004)Google Scholar
  28. 28.
    Ishibashi, Y., Tasaka, S.: A synchronization mechanism for continuous media in multimedia communications. In: IEEE Proc. 14th Annual Joint Conf. of the IEEE Computer and Communications Societies. Bringing Information to People (INFOCOM), vol. 3, pp. 1010–1019 (1995)Google Scholar
  29. 29.
    Tasaka, S., Nunome, T., Ishibashi, Y.: Live media synchronization quality of a retransmission-based error recovery scheme. In: Conference Record of IEEE ICC’00, pp. 1535–1541 (2000)Google Scholar
  30. 30.
    Fitzek, F., Reisslein, M.: Video traces for network performance evaluation: Yuv 4:2:0 video sequences, Accessed 13 Jan 2015
  31. 31.
    Silva, A., Ramirez, O., Vega, V., Oliver, J.: Phantom Omni haptic device: Kinematic and manipulability. In Electronics, Robotics and Automotive Mechanics Conf. (CERMA), pp. 193–198 (2009)Google Scholar
  32. 32.
    Fujimoto M., Ishibashi, Y.: Packetization Interval of haptic media in networked virtual environments. In: Proc. 4th ACM SIGCOMM Workshop on Network and System Support for Games, New York, pp. 1–6 (2005)Google Scholar
  33. 33.
    The Greek Research and Technology Network—GRNET S.A., Accessed 13 Jan 2015

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • George Kokkonis
    • 1
  • Kostas E. Psannis
    • 1
  • Manos Roumeliotis
    • 1
  • Yutaka Ishibashi
    • 2
  1. 1.Department of Applied InformaticsUniversity of MacedoniaThessalonikiGreece
  2. 2.Department of Scientific and Engineering SimulationNagoya Institute of TechnologyNagoyaJapan

Personalised recommendations