A Framework for Generation of Testsets for Recent Multimedia Workflows

  • Robert MantheyEmail author
  • Steve Conrad
  • Marc Ritter
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9739)


Our framework offers solution approaches for that inadequacy to be overcome. An abstract description define each test case, its transformation to the designated target platforms as well as the operations and parameters to be processed within the evaluation in such a way that it is independent of any platform. The control of our automated framework workflow is based on Python and Apache Ant which trigger the execution of the described definitions with the result that different tools can be used flexible and purpose-dependent.

We conduct an visual error detection evaluation of FFmpeg, Telestream Episode and Adobe Media Encoder. This consists the creation of single uncompressed images based on the definitions of the test patterns in POV-Ray. After that, they are merged together to video samples which form the platform dependent instances of the test cases. All of these videos are processed with different codecs and encoding qualities during the evaluation. The results are compared with its uncompressed raw material or other test cases.

The evaluation shows that the identical test case video file results in visual strongly different outcomes after the encoding. Furthermore some created test cases cause complete losses of the raw information data, ringing artefacts at contrast edges and flicker effects.


Framework Multimedia Quality analysis Testing 



This work was partially accomplished within the project localizeIT (funding code 03IPT608X) funded by the Federal Ministry of Education and Research (Bundesministerium für Wissenschaft und Forschung, Germany) in the program of Entrepreneurial Regions InnoProfile-Transfer.


  1. 1.
    Davies, E.: Machine Vision. Morgan Kaufmann, San Francisco (2005)Google Scholar
  2. 2.
    Manthey, R., Herms, R., Ritter, M., Storz, M., Eibl, M.: A support framework for automated video and multimedia workflows for production and archive. In: Yamamoto, S. (ed.) HCI 2013, Part III. LNCS, vol. 8018, pp. 336–341. Springer, Heidelberg (2013)Google Scholar
  3. 3.
    Ritter, M.: Optimization of algorithms for video analysis: a framework to fit the demands of local television stations. In: Eibl, M. (ed.) Wissenschaftliche Schriftenreihe Dissertationen der Medieninformatik, vol. 3, pp. i-xlii, 1–336. Universitätsverlag der Technischen Universität Chemnitz, Germany (2014).
  4. 4.
    Wiegand, T., Sullivan, G.J., Bjntegaard, G., Luthra, A.: Overview of the h.264/avc video coding standard. IEEE Trans. Circuits Syst. Video Technol. 13(7), 560–576 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Computer Science Junior Professorship Media ComputingTechnische Universität ChemnitzChemnitzGermany

Personalised recommendations