Memory Effects in Subjective Quality Assessment of X-Ray Images

  • Victor Landre
  • Marius Pedersen
  • Dag Waaler
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10270)


Experiments with human observers is considered as the most precise way for the assessment of image quality. Although widely used, such experiments have its pitfalls and hazards. In this work we investigate if the quality rating of previously viewed images influence the rating given to the current image, which we refer to as the rating memory effect. A subjective experiment with a group of observers rating x-ray images of different radiation dose was used for the basis of the analysis. The results indicate a memory effect, meaning that the rating of an image can be influenced by the ratings given in previously judged images.


X-ray Memory Subjective experiments Psychometrics Quality assessment 



We would like to thank Dr. Helle Precht, who provided the different images for the experiment.


  1. 1.
    Pedersen, M., Hardeberg, J.Y.: Full-reference image quality metrics: classification and evaluation. Found. Trends® Comput. Graph. Vis. 7(1), 1–80 (2012)CrossRefzbMATHGoogle Scholar
  2. 2.
    Engeldrum, P.G.: Psychometric Scaling: A Toolkit for Imaging Systems Development. Imcotek press, Winchester (2000)Google Scholar
  3. 3.
    Hamid, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)CrossRefGoogle Scholar
  4. 4.
    Ehrenstein, W.H., Ehrenstein, A.: Psychophysical methods. In: Windhorst, U., Johansson, H. (eds.) Modern Techniques in Neuroscience Research, pp. 1211–1241. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  5. 5.
    Engeldrum, P.G.: Psychometric scaling: avoiding the pitfalls and hazards. In: PICS, pp. 101–107 (2001)Google Scholar
  6. 6.
    Engeldrum, P.G.: Image quality modeling: Where are we? In: PICS, pp. 251–255 (1999)Google Scholar
  7. 7.
    International Atomic Energy Agency: Quality Assurance Programme for Computed Tomography: Diagnostic and Therapy Applications. Number 19 in IAEA Human Health Series (2012)Google Scholar
  8. 8.
    Hoßfeld, T., Biedermann, S., Schatz, R., Platzer, A., Egger, S., Fiedler, M.: The memory effect and its implications on web QoE modeling. In: Proceedings of the 23rd International Teletraffic Congress, pp. 103–110. International Teletraffic Congress (2011)Google Scholar
  9. 9.
    Freeman, J., Simoncelli, E.P.: Metamers of the ventral stream. Nat. Neurosci. 14(9), 1195–1201 (2011)CrossRefGoogle Scholar
  10. 10.
    Cohen, M.A., Dennett, D.C., Kanwisher, N.: What is the bandwidth of perceptual experience? Trends Cogn. Sci. 20(5), 324–335 (2016)CrossRefGoogle Scholar
  11. 11.
    Aideyan, U.O., Berbaum, K., Smith, W.L.: Influence of prior radiologic information on the interpretation of radiographic examinations. Acad. Radiol. 2(3), 205–208 (1995)CrossRefGoogle Scholar
  12. 12.
    Hardesty, L.A., Ganott, M.A., Hakim, C.M., Cohen, C.S., Clearfield, R.J., Gur, D.: “Memory effect” in observer performance studies of mammograms1. Acad. Radiol. 12(3), 286–290 (2005)CrossRefGoogle Scholar
  13. 13.
    Haygood, T.M., Liu, M.A.Q., Galvan, E.M., Bassett, R., Devine, C., Lano, E., Viswanathan, C., Marom, E.M.: Memory for previously viewed radiographs and the effect of prior knowledge of memory task. Acad. Radiol. 20(12), 1598–1603 (2013)CrossRefGoogle Scholar
  14. 14.
    Dugay, F., Farup, I., Hardeberg, J.Y.: Perceptual evaluation of color gamut mapping algorithms. Color Res. Appl. 33(6), 470–476 (2008)CrossRefGoogle Scholar
  15. 15.
    Bonnier, N., Schmitt, F., Brettel, H., Berche, S.: Evaluation of spatial gamut mapping algorithms. In: Color and Imaging Conference, vol. 2006, pp. 56–61. Society for Imaging, Science and Technology (2006)Google Scholar
  16. 16.
    Le Moan, S., Pedersen, M., Farup, I., Blahová, J.: The influence of short-term memory in subjective image quality assessment. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 91–95, September 2016Google Scholar
  17. 17.
    Aldridge, R., Davidoff, J., Ghanbari, M., Hands, D., Pearson, D.: Recency effect in the subjective assessment of digitally-coded television pictures. In: 5th International Conference on Image Processing and its Applications, pp. 336–339, July 1995Google Scholar
  18. 18.
    Hands, D.S., Avons, S.E.: Recency and duration neglect in subjective assessment of television picture quality. Appl. Cogn. Psychol. 15(6), 639–657 (2001)CrossRefGoogle Scholar
  19. 19.
    Seferidis, V., Ghanbari, M., Pearson, D.E.: Forgiveness effect in subjective assessment of packet video. Electron. Lett. 28(21), 2013–2014 (1992)CrossRefGoogle Scholar
  20. 20.
    Pinson, M.H., Wolf, S.: Comparing subjective video quality testing methodologies. In: Visual Communications and Image Processing 2003, pp. 573–582. International Society for Optics and Photonics (2003)Google Scholar
  21. 21.
    Huynh-Thu, Q., Garcia, M.-N., Speranza, F., Corriveau, P., Raake, A.: Study of rating scales for subjective quality assessment of high-definition video. IEEE Trans. Broadcast. 57(1), 1–14 (2011)CrossRefGoogle Scholar
  22. 22.
    Ickin, S., Janowski, L., Wac, K., Fiedler, M.: Studying the challenges in assessing the perceived quality of mobile-phone based video. In: 2012 Fourth International Workshop on Quality of Multimedia Experience (QoMEX), pp. 164–169. IEEE (2012)Google Scholar
  23. 23.
    Hamberg, R., de Ridder, H.: Time-varying image quality: modeling the relation between instantaneous and overall quality. SMPTE J. 108(11), 802–811 (1999)CrossRefGoogle Scholar
  24. 24.
    Klapproth, F.: Time and decision making in humans. Cogn. Affect. Behav. Neurosci. 8(4), 509–524 (2008)CrossRefGoogle Scholar
  25. 25.
    CIE: Guidelines for the evaluation of gamut mapping algorithms. Commision Internationale De L’eclairage, vol. 153, pp. D8–6 (2003)Google Scholar
  26. 26.
    Precht, H., Gerke, O., Rosendahl, K., Tingberg, A., Waaler, D.: Digital radiography: optimization of image quality and dose using multi-frequency software. Pediatr. Radiol. 42(9), 1112–1118 (2012)CrossRefGoogle Scholar
  27. 27.
    Precht, H., Gerke, O., Rosendahl, K., Tingberg, A., Waaler, D.: Large dose reduction by optimization of multifrequency processing software in digital radiography at follow-up examinations of the pediatric femur. Pediatr. Radiol. 44(2), 239 (2014)CrossRefGoogle Scholar
  28. 28.
    Van Ngo, K., Storvik, J. Jr., Dokkeberg, C.A., Farup, I., Pedersen, M.: Quickeval: a web application for psychometric scaling experiments. In: Larabi, M.-C., Triantaphillidou, S. (eds.) Image Quality and System Performance XI, vol. 9396, p. 93960O (2015)Google Scholar
  29. 29.
    ITUR Rec.: Bt. 500–11 (2002). Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, 39 (2009)Google Scholar
  30. 30.
    Martensen, K.M.Q.: Radiographic Image Analysis. Elsevier Health Sciences, Amsterdam (2013)Google Scholar
  31. 31.
    Box, G.E., Jenkins, G.M.: Time Series Analysis: Forecasting and Control, revised edn. Holden-Day, San Francisco (1976)zbMATHGoogle Scholar
  32. 32.
    Box, G.E., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control Revised Ed. Wiley, Hoboken (2015)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Norwegian University of Science and TechnologyGjøvikNorway

Personalised recommendations