Skip to main content
SpringerLink
Log in

The impact of video encoding parameters on QoE of simulated FPV drone control

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Applications of Unmanned Aerial Systems (UAS) have become increasingly diverse and advanced over the years. While initial applications targeted strictly military applications, with a rapid decrease in cost and increase in availability to consumers, applications such as surveillance, journalism, agriculture, film-making, entertainment, etc. have begun to flourish. One of these applications is an immersive drone-piloting modality known as First-person view (FPV) piloting. This method of flying leads to the challenge of streaming video and transmitting control information at very low latencies to enable proper real-time control. FPV technology most commonly relies on implementing video streaming through analog technology due to strict latency requirements. In this paper, we present our research directed toward developing a digital alternative to this technology. We report on two user studies designed to assess the Quality of Experience (QoE) of an FPV system, coupled with a cloud-based drone flight simulator, with the objective to evaluate the impact of various video encoding parameters, namely bitrate, resolution, and frame rate. Two user studies with the same core test methodology were conducted: the first involved University students and staff at the Univ. of Zagreb, while the second study involved Spanish Air Force Academy flight students and instructors with (unlike the first study group) significant prior experience in piloting manned aircraft and experience in the use of a flight simulator for military pilot training. The primary motivation of conducting the second user study was thus to investigate quality perception involving users with different prior experience as compared to our first study. Obtained results indicate that video encoding parameters have a significant impact on user perceived QoE, with only a slight impact of prior piloting experience on quality ratings. Furthermore, our findings highlight the relevance of simulator sickness in this kind of system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Availability of data and materials

The anonymized datasets are made publicly available via an open repository. Please see details at https://muexlab.fer.hr/muexlab/research/datasets.

Code Availability

Not applicable.

Notes

  1. https://skydive.orqafpv.com/

  2. https://store.steampowered.com/

  3. https://store.steampowered.com/remoteplay

  4. https://orqafpv.com/fpvone

  5. https://dronenodes.com/orqa-fpv-one-goggles-review/

  6. https://store.nvidia.com/en-us/shield/

  7. https://betafpv.com/products/literadio-2-radio-transmitter

  8. Link to be provided pending paper acceptance

References

  1. Baltaci A, Dinc E, Ozger M, Alabbasi A, Cavdar C, Schupke D (2021) A survey of wireless networks for future aerial communications (facom). IEEE Commun Surv Tutor

  2. Brunnström K, Dima E, Qureshi T, Johanson M, Andersson M, Sjöström M (2020) Latency impact on quality of experience in a virtual reality simulator for remote control of machines. Signal Process Image Commun 89:116005

    Article  Google Scholar 

  3. Ji H, Park S, Yeo J, Kim Y, Lee J, Shim B (2017) Introduction to ultra reliable and low latency communications in 5G. arXiv:1704.05565

  4. Barin A, Dolgov I, Toups ZO (2017) Understanding dangerous play: a grounded theory analysis of high-performance drone racing crashes. In: Proceedings of the annual symposium on computer-human interaction in play, pp. 485–496

  5. Chan K, Nirmal U, Cheaw W (2018) Progress on drone technology and their applications: A comprehensive review. In: AIP conference proceedings, vol. 2030, p. 020308. AIP Publishing LLC

  6. Kovačević S, Matijević V, Jelušić I, Keser T (2017) High-definition video goggles for unmanned aerial vehicle operation and other remote operation applications. In: 27th International scientific and professional conference”organization and maintenance technology”, p. 183

  7. OSpreyFPVracing: Digital FPV (2019) Is it the future of drone racing? https://www.getfpv.com/learn/fpv-essentials/digital-fpv-is-it-the-future/

  8. Celentano U, Kauppinen M, Röning J (2021) Ground support for drone-based industrial inspections. In: 2021 Aerial robotic systems physically interacting with the environment (AIRPHARO), pp. 1–5. IEEE

  9. OSpreyFPVracing (2018) All about analogue FPV video and the clearview’s magic. https://www.getfpv.com/learn/fpv-in-depth-and-technical/analogue-fpv-video/

  10. Stanco F, Allegra D, Milotta FLM (2016) Tracking error in digitized analog video: automatic detection and correction. Multimed Tools Appl 75(7):3733–3746

    Article  Google Scholar 

  11. Marpe D, Wiegand T, Sullivan GJ (2006) The h. 264/mpeg4 advanced video coding standard and its applications. IEEE Commun Mag 44(8):134–143

  12. Bertizzolo L, Tran TX, Buczek J, Balasubramanian B, Jana R, Zhou Y, Melodia T (2021) Streaming from the air: Enabling drone-sourced video streaming applications on 5g open-ran architectures. IEEE Trans Mob Comput

  13. Jahromi HZ, Bartolec I, Gamboa E, Hines A, Schatz R (2020) You drive me crazy! Interactive QoE assessment for telepresence robot control. In: 2020 12th International conference on quality of multimedia experience (QoMEX), pp. 1–6

  14. Dima E, Brunnström K, Sjöström M, Andersson M, Edlund J, Johanson M, Qureshi T (2019) View position impact on QoE in an immersive telepresence system for remote operation. In: 2019 11th Int’l conf. on quality of multimedia experience (QoMEX), pp. 1–3

  15. Aykut T, Lochbrunner S, Karimi M, Cizmeci B, Steinbach E (2017) A stereoscopic vision system with delay compensation for 360 remote reality. In: Proc. of the thematic workshops of ACM multimedia 2017, pp. 201–209

  16. Orlosky J, Kiyokawa K, Takemura H (2017) Virtual and augmented reality on the 5G highway. J Inf Process 25:133–141

    Google Scholar 

  17. Baltaci A, Cech H, Mohan N, Geyer F, Bajpai V, Ott J, Schupke D (2022) Analyzing real-time video delivery over cellular networks for remote piloting aerial vehicles. In: Proceedings of the 22nd ACM internet measurement conference, pp. 98–112

  18. Nightingale J, Salva-Garcia P, Calero JMA, Wang Q (2018) 5G-QoE: QoE modelling for ultra-HD video streaming in 5G networks. IEEE Trans Broadcast 64(2):621–634

    Article  Google Scholar 

  19. Skorin-Kapov L, Varela M, Hoßfeld T, Chen K-T (2018) A survey of emerging concepts and challenges for qoe management of multimedia services. ACM Trans Multimed Comput Commun Appl (TOMM) 14(2s):1–29

    Google Scholar 

  20. Barakabitze AA, Barman N, Ahmad A, Zadtootaghaj S, Sun L, Martini MG, Atzori L (2019) Qoe management of multimedia streaming services in future networks: a tutorial and survey. IEEE Commun Surv Tutor 22(1):526–565

    Article  Google Scholar 

  21. Perkis A, Timmerer C et al (2020) QUALINET white paper on definitions of immersive media experience (IMEx). European network on quality of experience in multimedia systems and services, 14th QUALINET meeting (online)

  22. Silic M, Suznjevic M, Skorin-Kapov L (2021) Qoe assessment of fpv drone control in a cloud gaming based simulation. In: 2021 13th International conference on quality of multimedia experience (QoMEX), pp. 175–180. IEEE

  23. Wu E (2017) Fly it like you mean it mitigating motion sickness in first-person-view drones. EE267: Virtual reality

  24. Suznjevic M, Skorin-Kapov L, Matijasevic M (2013) The impact of user, system, and context factors on gaming qoe: a case study involving mmorpgs. In: 2013 12th Annual workshop on network and systems support for games (NetGames), pp. 1–6. IEEE

  25. Seo J, Duque L, Wacker J (2018) Drone-enabled bridge inspection methodology and application. Autom Constr 94:112–126

    Article  Google Scholar 

  26. Shi W, Zhou H, Li J, Xu W, Zhang N, Shen X (2018) Drone assisted vehicular networks: Architecture, challenges and opportunities. IEEE Netw 32(3):130–137

    Article  Google Scholar 

  27. Kaleem Z, Rehmani MH (2018) Amateur drone monitoring: State-of-the-art architectures, key enabling technologies, and future research directions. IEEE Wirel Commun 25(2):150–159

    Article  Google Scholar 

  28. Galvane Q, Fleureau J, Tariolle F-L, Guillotel P (2017) Automated cinematography with unmanned aerial vehicles. arXiv:1712.04353

  29. González Serrato N, Solera Delgado M, Ruiz Vega F, Gijón Martín C, Toril Genovés M (2022) A quality of experience evaluation methodology for first-person-view drone control in cellular networks. In: Proceedings of the 19th ACM international symposium on performance evaluation of wireless ad hoc, sensor, & ubiquitous networks, pp. 59–66

  30. Boban L, Catania L, Allegra D, Suznjevic M (2023) Investigating the effect of region of interest coding on the qoe of fpv drone piloting under adverse network conditions. In: 2023 17th International conference on telecommunications (ConTEL), pp. 1–8. IEEE

  31. Pfeiffer C, Scaramuzza D (2021) Human-piloted drone racing: Visual processing and control. IEEE Robot Autom Lett 6(2):3467–3474

    Article  Google Scholar 

  32. Song Y, Steinweg M, Kaufmann E, Scaramuzza D (2021) Autonomous drone racing with deep reinforcement learning. In: 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 1205–1212. IEEE

  33. Tezza D, Laesker D, Andujar M (2021) The learning experience of becoming a fpv drone pilot. In: Companion of the 2021 ACM/IEEE international conference on human-robot interaction, pp. 239–241

  34. Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993) Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 3(3):203–220

    Article  Google Scholar 

  35. Smolyanskiy N, Gonzalez-Franco M (2017) Stereoscopic first person view system for drone navigation. Front Robot AI 4:11

    Article  ADS  Google Scholar 

  36. Kim D-H, Go Y-G, Choi S-M (2020) An aerial mixed-reality environment for first-person-view drone flying. Appl Sci 10(16):5436

    Article  CAS  Google Scholar 

  37. Asan A, Robitza W, Mkwawa I-h, Sun L, Ifeachor E, Raake A (2017) Impact of video resolution changes on qoe for adaptive video streaming. In: 2017 IEEE international conference on multimedia and expo (ICME), pp. 499–504. IEEE

  38. Li J, Feng R, Liu Z, Sun W, Li Q (2018) Modeling QoE of virtual reality video transmission over wireless networks. In: 2018 IEEE GLOBECOM, pp. 1–7

  39. Kacianka S, Hellwagner H (2015) Adaptive video streaming for uav networks. In: Proceedings of the 7th ACM international workshop on mobile video, pp. 25–30

  40. Benjak J, Hofman D, Knezović J, Žagar M (2022) Performance comparison of h. 264 and h. 265 encoders in a 4k fpv drone piloting system. Appl Sci 12(13):6386

  41. Kimura N, Kono M, Rekimoto J (2019) Deep dive: Deep-neural-network-based video extension for immersive head-mounted display experiences. In: Proceedings of the 8th ACM international symposium on pervasive displays, pp. 1–7

  42. Slivar I, Suznjevic M, Skorin-Kapov L (2018) Game categorization for deriving qoe-driven video encoding configuration strategies for cloud gaming. ACM Trans Multimed Comput Commun Appl 14(3s):1–24

    Article  Google Scholar 

  43. Metzger F, Geißler S, Grigorjew A, Loh F, Moldovan C, Seufert M, Hoßfeld T (2022) An introduction to online video game qos and qoe influencing factors. IEEE Commun Surv Tutor 24(3):1894–1925

    Article  Google Scholar 

  44. Zadtootaghaj S, Schmidt S, Möller S (2018) Modeling gaming QoE: Towards the impact of frame rate and bit rate on cloud gaming. In: 2018 10th Int’l conf. on quality of multimedia experience (QoMEX)

  45. Slivar I, Skorin-Kapov L, Suznjevic M (2016) Cloud gaming qoe models for deriving video encoding adaptation strategies. In: Proceedings of the 7th MMSys’16, pp. 1–12

  46. Mairaj A, Baba AI, Javaid AY (2019) Application specific drone simulators: Recent advances and challenges. Simul Model Pract Theory 94:100–117

    Article  Google Scholar 

  47. ITU-T Recom (1999) P.910: Subjective video quality assessment methods for multimedia applications. International telecommunication union

  48. Balk SA, Bertola MA, Inman VW (2013) Simulator sickness questionnaire: Twenty years later

  49. ITU-T Recommendation (2022) P.910 Subjective video quality assessment methods for multimedia applications. Technical report, ITU-T

  50. Steamworks (2021) Steam - 2020 year in review. https://store.steampowered.com/news/group/4145017/view/2961646623386540826

  51. Choi CH, Jang HJ, Lim SG, Lim HC, Cho SH, Gaponov I (2016) Automatic wireless drone charging station creating essential environment for continuous drone operation. In: 2016 Int’l conf on control, automation and information sciences (ICCAIS), pp. 132–136

  52. Molina J, Muelas D, De Vergara JEL, García-Aranda JJ (2020) Network quality-aware architecture for adaptive video streaming from drones. IEEE Internet Comput 24(1):5–13

    Article  Google Scholar 

  53. Theolin H (2018) Video compression optimized for racing drones, Master’s Degree Project. Technical report, Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering

Download references

Funding

This work has been supported in part by the project KK.01.2.1.02.0054 Razvoj uredaja za prijenos video signala ultra niske latencije (Development of ultra low latency video signal transmission device), financed by the EU from the European Regional Development Fund, and by the Croatian Science Foundation under the project Modeling and Monitoring QoE for Immersive 5G-Enabled Multimedia Services (Q-MERSIVE), grant number IP-2019-04-9793.

Author information

Authors and Affiliations

  1. University of Zagreb Faculty of Electrical Engineering and Computing, Zagreb, Croatia

    Matko Šilić, Mirko Sužnjević & Lea Skorin-Kapov

  2. University Center of Defense, San Javier Air Force Base, MDE-UPCT, Murcia, Spain

    Nina Skorin-Kapov & Marcelo Izquierdo Lorenzana

Authors
  1. Matko Šilić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mirko Sužnjević
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lea Skorin-Kapov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nina Skorin-Kapov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marcelo Izquierdo Lorenzana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: L. Skorin-Kapov, M. Sužnjević, N. Skorin-Kapov; Implementation: M. Šilić; Study execution: M. Šilić, N. Skorin-Kapov, M. Izquierdo Lorenzana; Data analysis: M. Šilić, M. Izquierdo Lorenzana; Writing: M. Šilić, L. Skorin-Kapov; Editing: M. Sužnjević, N. Skorin-Kapov; Funding aquisition: L. Skorin-Kapov, M. Sužnjević. All authors read and approved the final manuscript. We confirm that the order of authors has been approved by all named authors.

Corresponding author

Correspondence to Lea Skorin-Kapov.

Ethics declarations

Conflict of interest/Competing interests

The authors confirm there are no known conflicts of interest/competing interests associated with this paper that could inappropriately influence, or be perceived to influence, this work.

Ethics approval

The study received necessary approval of the responsible Ethics Committee.

Consent to participate

The authors confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed.

Consent for publication

All authors agree with the content and give explicit consent to submit the paper. Consent was also obtained from the responsible authorities at the institute/organization where the work has been carried out, before the work was submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Šilić, M., Sužnjević, M., Skorin-Kapov, L. et al. The impact of video encoding parameters on QoE of simulated FPV drone control. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18442-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18442-2

Keywords

Search

Navigation