Abstract
Nowadays, the majority of Internet traffic is multimedia content. Video streaming services are in high demand by end users and use HTTP Adaptive Streaming (HAS) as transmission protocol. HAS splits the video into non-overlapping chunks and each video chunk can be encoded independently using different representations. Therefore, these encode tasks can be parallelized and Cloud computing can be used for this. However, in the most extended solutions, the infrastructure must be configured and provisioned in advance. Recently, serverless platforms have made posible to deploy functions that can scale from zero to a configurable maximum. This work presents and analyses the behavior of event-driven serverless functions to encode video chunks and to compute, optionally, the quality of the encoded videos. These functions have been implemented using an adapted version of embedded Tomcat to deal with CloudEvents. We have deployed these event-driven serverless pipelines for video coding and quality metrics on an on-premises serverless platform based on Knative on one master node and eight worker nodes. We have tested the scalability and resource consumption of the proposed solution using two video codecs: x264 and AV1, varying the maximum number of replicas and the resources allocated to them (fat and slim function replicas). We have encoded different 4K videos to generate multiple representations per function call and we show how it is possible to create pipelines of serverless media functions. The results of the different tests carried out show the good performance of the serverless functions proposed. The system scales the replicas and distributes the jobs evenly across all the replicas. The overall encoding time is reduced by 18% using slim replicas but fat replicas are more adequate in live video streaming as the encoding time per chunk is reduced. Finally, the results of the pipeline test show an appropriate distribution and chaining among the available replicas of each function type.
Availability of Data and Material
Data sharing not applicable to this article as no datasets were generated or analysed during the current study.
References
Ao, L., Izhikevich, L., Voelker, G.M., Porter, G.: Sprocket: a serverless video processing framework. In: Proceedings of the ACM Symposium on Cloud Computing, SoCC ’18, pp. 263–274. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3267809.3267815 (2018)
CloudEvents: Cloudevents project. https://cloudevents.io/. Accessed 01 Aug 2022 (2022)
Dong, Y., Zhang, X., Zhao, Y., Song, L.: A containerized media cloud for video transcoding service. In: 2018 IEEE International Conference on Consumer Electronics (ICCE), pp. 1–4. https://doi.org/10.1109/ICCE.2018.8326347 (2018)
FFmpeg: Ffmpeg tool. https://ffmpeg.org/ . Accessed 01 Aug 2022 (2022)
Fouladi, S., Wahby, R. S., Shacklett, B., Balasubramaniam, K. V., Zeng, W., Bhalerao, R., Sivaraman, A., Porter, G., Winstein, K.: Encoding, fast and slow: Low-latency video processing using thousands of tiny threads. In: 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17), pp. 363–376 (2017)
Guo, L., De Cock, J., Aaron, A.: Compression performance comparison of X264, X265, Libvpx and Aomenc for On-demand adaptive streaming applications. In: 2018 Picture Coding Symposium (PCS), pp. 26–30. https://doi.org/10.1109/PCS.2018.8456302 (2018)
Gutiérrez-Aguado, J.: Adapting embeded Tomcat to develop event-driven serverless functions. In: JCIS2022. SISTEDES. http://hdl.handle.net/11705/JCIS/2022/040 (2022)
Gutiérrez-Aguado, J., Peña-Ortiz, R., Garcia-Pineda, M., Claver, J.M.: A cloud-based distributed architecture to accelerate video encoders. Applied Sciences 10(15). https://doi.org/10.3390/app10155070. https://www.mdpi.com/2076-3417/10/15/5070 (2020)
Gutiérrez-Aguado, J., Peña-Ortiz, R., García-Pineda, M., Claver, J.M.: Cloud-based elastic architecture for distributed video encoding: evaluating H.265, VP9, and AV1. J. Netw. Comput. Appl. 171, 102782 (2020). https://doi.org/10.1016/j.jnca.2020.102782. https://www.sciencedirect.com/science/article/pii/S1084804520302563
IBM Watson Media Support Center: Internet connection and recommended encoding settings – IBM Watson Media. https://support.video.ibm.com/hc/en-us/articles/207852117-Internet-connection-and-recommended-encoding-settings
Jangda, A., Pinckney, D., Brun, Y., Guha, A.: Formal foundations of serverless computing. Proc. ACM Program. Lang. 3(OOPSLA). https://doi.org/10.1145/3360575 (2019)
Jeon, M., Kim, N., Lee, B.: Mapreduce-based distributed video encoding using content-aware video segmentation and scheduling. IEEE Access 4, 6802–6815 (2016). https://doi.org/10.1109/ACCESS.2016.2616540
Kemp, S.: Digital 2022: Global Overview Report. https://datareportal.com/reports/digital-2022-global-overview-report (2022)
Kerdranvat, M., Chen, Y., Jullian, R., Galpin, F., François, E.: The video codec landscape in 2020. ITU Journal: ICT Discoveries 3(1), 73–83 (2020). http://handle.itu.int/11.1002/pub/8153d78c-en
Kim, M., Cui, Y., Han, S., Lee, H.: Towards efficient design and implementation of a hadoop-based distributed video transcoding system in cloud computing environment. Int. J. Multimed. Ubiquit. Eng. 8(2), 213–224 (2013). https://doi.org/10.14257/ijmue.2013.8.2.20
Knative: Knative is an open-source enterprise-level solution to build serverless and event driven applications. https://knative.dev/docs/. Accessed 01 Aug 2022 (2022)
Lederer, S.: Optimal adaptive streaming formats mpeg-dash & hls segment length. Bitmovin . https://bitmovin.com/mpeg-dash-hls-segment-length/ (2020)
Li, J., Kulkarni, S.G., Ramakrishnan, K.K., Li, D.: Analyzing open-source serverless platforms: characteristics and performance. International Conferences on Software Engineering and Knowledge Engineering . https://doi.org/10.18293/seke2021-129. (2021)
Li, Z., Aaron, A., Katsavounidis, I., Moorthy, A., Manohara, M.: Toward a practical perceptual video quality metric. The Netflix Tech. Blog 6(2) (2016)
Li, Z., Duanmu, Z., Liu, W., Wang, Z.: AVC, HEVC, VP9, AVS2 or AV1? - a comparative study of state-of-the-art video encoders on 4K videos. In: ICIAR (2019)
Marathe, N., Gandhi, A., Shah, J. M.: Docker Swarm and Kubernetes in cloud computing environment. In: 2019 3Rd International Conference on Trends in Electronics and Informatics (ICOEI), pp. 179–184 (2019), https://doi.org/10.1109/ICOEI.2019.8862654
Martins, H., Araujo, F., da Cunha, P. R.: Benchmarking serverless computing platforms. Journal of Grid Computing 18(4), 691–709 (2020). https://doi.org/10.1007/s10723-020-09523-1
Moravcik, M., Kontsek, M.: Overview of docker container orchestration tools. In: 2020 18th International Conference on Emerging Elearning Technologies and Applications (ICETA), pp. 475–480. https://doi.org/10.1109/ICETA51985.2020.9379236 (2020)
Pereira, R., Azambuja, M., Breitman, K., Endler, M.: An architecture for distributed high performance video processing in the cloud. In: 2010 IEEE 3Rd International Conference on Cloud Computing, pp. 482–489 (2010), https://doi.org/10.1109/CLOUD.2010.73
Pääkkönen, P., Heikkinen, A., Aihkisalo, T.: Architecture for predicting live video transcoding performance on docker containers. In: 2018 IEEE International Conference on Services Computing (SCC), pp. 65–72 (2018), https://doi.org/10.1109/SCC.2018.00016
Risco, S., Moltó, G., Naranjo, D.M., Blanquer, I.: Serverless workflows for containerised applications in the cloud continuum. Journal of Grid Computing 19(3), 30 (2021). https://doi.org/10.1007/s10723-021-09570-2
Sameti, S., Wang, M., Krishnamurthy, D.: Stride: distributed video transcoding in spark. In: 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC), pp. 1–8 (2018), https://doi.org/10.1109/PCCC.2018.8711214
Sameti, S., Wang, M., Krishnamurthy, D.: Contrast: container-based transcoding for interactive video streaming. In: NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium, pp. 1–9 (2020), https://doi.org/10.1109/NOMS47738.2020.9110469
Seufert, M., Egger, S., Slanina, M., Zinner, T., Hoßfeld, T., Tran-Gia, P.: A survey on quality of experience of http adaptive streaming. IEEE Communications Surveys & Tutorials 17(1), 469–492 (2014)
Sharma, P., Chaufournier, L., Shenoy, P., Tay, Y.C.: Containers and virtual machines at scale: a comparative study. In: Proceedings of the 17th International Middleware Conference, Middleware ’16. Association for Computing Machinery, New York, NY, USA (2016), https://doi.org/10.1145/2988336.2988337
Song, C., Shen, W., Sun, L., Lei, Z., Xu, W.: Distributed video transcoding based on mapreduce. In: 2014 IEEE/ACIS 13th International Conference on Computer and Information Science (ICIS), pp. 309–314 (2014), https://doi.org/10.1109/ICIS.2014.6912152
Taibi, D., Spillner, J., Wawruch, K.: Serverless computing-where are we now, and where are we heading? IEEE Softw. 38(1), 25–31 (2021). https://doi.org/10.1109/MS.2020.3028708
Wang, L., Li, M., Zhang, Y., Ristenpart, T.: Swift, M.: Peeking behind the curtains of serverless platforms. In: 2018 USENIX Annual Technical Conference (USENIX ATC 18), pp. 133–146. USENIX Association, Boston, MA. https://www.usenix.org/conference/atc18/presentation/wang-liang (2018)
Acknowledgments
Not applicable.
Funding
Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been supported by PID2021-126209 OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”, by the European Union and by UV-INV-AE-1564749 funded by the University of Valencia.
Author information
Authors and Affiliations
Contributions
All authors contributed to the conception and design of the study. Material preparation, experimental design and data collection were performed by Wilmer Moina-Rivera and Juan Gutiérrez-Aguado. Software implementation was performed by Juan Gutiérrez-Aguado and Wilmer Moina-Rivera. The analysis of the results was carried out by all authors. The first draft of the manuscript was written by all authors, and all authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflict of interest.
Informed Consent
Not applicable.
Consent for Publication
Not applicable.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Moina-Rivera, W., Garcia-Pineda, M., Claver, J.M. et al. Event-Driven Serverless Pipelines for Video Coding and Quality Metrics. J Grid Computing 21, 20 (2023). https://doi.org/10.1007/s10723-023-09647-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10723-023-09647-0
Keywords
- Serverless
- Function as a service
- CloudEvents
- Video coding
- Quality metrics
- HTTP adaptive streaming