Scoring Photographic Rule of Thirds in a Large MIRFLICKR Dataset: A Showdown Between Machine Perception and Human Perception of Image Aesthetics
In this research we have developed and evaluated a system that uses the image compositional metric called ‘Rule of Thirds’ used by photographers to grade visual aesthetics of an image. The novel aspect of the work is that it combines quantitative and qualitative aspects of research by taking human psychology into account. The core idea is to identify how similar the perception of a ‘good image’ and ‘bad image’ is by machines versus humans (through a user study based on 255 participants on 5000 images from the standard MIRFLICKR database ). We have considered the compositional norm, namely ‘rule of thirds’ used by photographers and inspired by the golden ratio that states that - if an image is segmented on a 3 × 3 grid, then it is appealing to the eye when the most salient object(s) or ‘subject(s)’ of the image is located precisely on or aligned on the middle grid lines . First, we preprocess the input image by labeling the regions of attraction for human eye using two saliency algorithms namely Graph-Based Visual Saliency (GBVS)  and Itti-Koch . Next, we quantify the rule of thirds property in images by mathematically considering the location of salient region(s) adhering to rule of thirds. This is then used to rank or score an input image. To validate, we conducted a user study where 255 human subjects ranked the images and compared our algorithmic results, making it a both a quantitative and qualitative research. We have also analyzed and presented the performance differences between two saliency algorithms and presented ROC plots along with similarity quantification between algorithms and human subjects. Our massive user study and experimental results provides the evidence of modern machine’s ability to mimic human-like behavior. Along with it, results computationally prove significance of rule of thirds.
KeywordsImage processing Visual perception Image saliency Rule of thirds Photography Golden ratio Computer vision Image score Image composition Image Flickr
This work was funded by North South University’s annual research grant for the fiscal year 2017–18. We would like to thank Professor John Kender (http://www.cs.columbia.edu/~jrk/), Professor of Computer Science at Columbia University for his insights.
- 1.Mai, L., Le, H., Niu, Y., Liu, F.: Rule of thirds detection from photograph. In: Proceedings of the IEEE International Symposium on Multimedia (ISM), Dana Point, CA, USA, pp. 91–96 (2011)Google Scholar
- 2.Peterson, B.: Learning to See Creatively, 1st edn, pp. 92–93. Amphoto Books, New York (2003)Google Scholar
- 3.Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2006)Google Scholar
- 5.Mai, L., Le, H., Niu, Y., Liu, F.: Rule of thirds detection from photograph. In: IEEE International Symposium on Multimedia, Portland (2011)Google Scholar
- 7.Maleš, M., Heđi, A., Grgić, M.: Compositional rule of thirds detection. In: IEEE International Symposium on ELMAR 2011, Dana Point (2011)Google Scholar
- 9.Huiskes, M.J., Lew, M.S.: The MIR flickr retrieval evaluation. In: ACM International Conference on Multimedia Information Retrieval (MIR 2008), Vancouver, Canada (2008)Google Scholar
- 10.Weisstein, E.W.: Golden Ratio. From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/GoldenRatio.html
- 11.McCurry, S.: 9 Photo Composition Tips (feat. Steve McCurry). Youtube (2015). https://youtu.be/7ZVyNjKSr0M. Accessed 2 March 2016