Skip to main content

Memorability: An Image-Computable Measure of Information Utility

  • 813 Accesses

Abstract

The pixels in an image, and the objects, scenes, and actions that they compose, determine whether an image will be memorable or forgettable. While memorability varies by image, it is largely independent of an individual observer. Observer independence is what makes memorability an image-computable measure of information, and eligible for automatic prediction. In this chapter, we zoom into memorability with a computational lens, detailing the state-of-the-art algorithms that accurately predict image memorability relative to human behavioral data, using image features at different scales from raw pixels to semantic labels. We discuss the design of algorithms and visualizations for face, object, and scene memorability, as well as algorithms that generalize beyond static scenes to actions and videos. We cover the state-of-the-art deep learning approaches that are the current front runners in the memorability prediction space. Beyond prediction, we show how recent A.I. approaches can be used to create and modify visual memorability. Finally, we preview the computational applications that memorability can power, from filtering visual streams to enhancing augmented reality interfaces.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-81465-6_8
  • Chapter length: 33 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-81465-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  • Abdal, R., Qin, Y., & Wonka, P. (2019). Image2StyleGAN: How to embed images into the StyleGAN latent space? In 2019 IEEE/CVF international conference on computer vision (ICCV) (pp. 4431–4440). https://doi.org/10.1109/ICCV.2019.00453.

  • Akagunduz, E., Bors, A., & Evans, K. (2019). Defining image memorability using the visual memory schema. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(9), 2165–2178.

    Google Scholar 

  • Anirudh, R., Thiagarajan, J. J., Kailkhura, B., & Bremer, P. T. (2020). MimicGAN: Robust projection onto image manifolds with corruption mimicking. International Journal of Computer Vision, 128(10), 2459–2477. https://doi.org/10.1007/s11263-020-01310-5.

    Google Scholar 

  • Bainbridge, W., Isola, P., Blank, I., & Oliva, A. (2012). Establishing a database for studying human face photograph memory. In Proceedings of the annual meeting of the cognitive science society (vol. 34).

    Google Scholar 

  • Bainbridge, W. A., Isola, P., & Oliva, A. (2013). The intrinsic memorability of face photographs. Journal of Experimental Psychology: General, 142(4), 1323.

    Google Scholar 

  • Barsom, E. Z., Graafland, M., & Schijven, M. P. (2016). Systematic review on the effectiveness of augmented reality applications in medical training. Surgical Endoscopy, 30(10), 4174–4183.

    Google Scholar 

  • Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017). Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6541–6549).

    Google Scholar 

  • Bau, D., Strobelt, H., Peebles, W., Wulff, J., Zhou, B., Zhu, J. Y., & Torralba, A. (2019). Semantic photo manipulation with a generative image prior. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH), 38(4), 1–11. https://doi.org/10.1145/3306346.3323023.

    Google Scholar 

  • Baveye, Y., Cohendet, R., Perreira Da Silva, M., & Le Callet, P. (2016). Deep learning for image memorability prediction: The emotional bias. In Proceedings of the 24th ACM international conference on Multimedia (pp. 491–495).

    Google Scholar 

  • Borkin, M. A., Vo, A. A., Bylinskii, Z., Isola, P., Sunkavalli, S., Oliva, A., & Pfister, H. (2013). What makes a visualization memorable? IEEE Transactions on Visualization and Computer Graphics, 19(12), 2306–2315.

    Google Scholar 

  • Borkin, M. A., Bylinskii, Z., Kim, N. W., Bainbridge, C. M., Yeh, C. S., Borkin, D., Pfister, H., & Oliva, A. (2015). Beyond memorability: Visualization recognition and recall. IEEE Transactions on Visualization and Computer Graphics, 22(1), 519–528.

    Google Scholar 

  • Bradley, M. M., Greenwald, M. K., Petry, M. C., & Lang, P. J. (1992). Remembering pictures: pleasure and arousal in memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(2), 379.

    Google Scholar 

  • Brady, T. F., Konkle, T., Alvarez, G. A., & Oliva, A. (2008). Visual long-term memory has a massive storage capacity for object details. Proceedings of the National Academy of Sciences, 105(38), 14325–14329.

    Google Scholar 

  • Brady, T., Konkle, T., Alvarez, G., & Oliva, A. (2011). Are real-world objects represented as bound units? independent decay of object details from short-term to long-term memory. Journal of Vision, 11(11), 1289–1289.

    Google Scholar 

  • Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 1904–1920, 3(3), 296–322. https://doi.org/10.1111/j.2044-8295.1910.tb00207.x.

    Google Scholar 

  • Buchanan, T. W., & Adolphs, R. (2002). The role of the human amygdala in emotional modulation of long-term declarative memory. Advances in Consciousness Research, 44, 9–34.

    Google Scholar 

  • Bylinskii, Z., Isola, P., Bainbridge, C., Torralba, A., & Oliva, A. (2015). Intrinsic and extrinsic effects on image memorability. Vision Research, 116, 165–178.

    Google Scholar 

  • Cahill, L., & McGaugh, J. L. (1995). A novel demonstration of enhanced memory associated with emotional arousal. Consciousness and Cognition, 4(4), 410–421.

    Google Scholar 

  • Carreira, J., & Zisserman, A. (2017). Quo vadis, action recognition? A new model and the kinetics dataset. In: 2017 IEEE Conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21–26, 2017 (pp. 4724–4733). https://doi.org/10.1109/CVPR.2017.502.

  • Celikkale, B., Erdem, A., & Erdem, E. (2015). Predicting memorability of images using attention-driven spatial pooling and image semantics. Image and Vision Computing, 42, 35–46.

    Google Scholar 

  • Cohendet, R., Yadati, K., Duong, N. Q., & Demarty, C. H. (2018). Annotating, understanding, and predicting long-term video memorability. In: Proceedings of the 2018 ACM on international conference on multimedia retrieval (pp. 178–186). ACM.

    Google Scholar 

  • Cohendet, R., Demarty, C., Duong, N. Q. K., & Martin, E. (2019). VideoMem: Constructing, analyzing, predicting short-term and long-term video memorability. In Proceedings of the IEEE international conference on computer vision (pp. 2531–2540).

    Google Scholar 

  • Creswell, A., & Bharath, A. A. (2019). Inverting the generator of a generative adversarial network. IEEE Transactions on Neural Networks and Learning Systems, 30(7), 1967–1974. https://doi.org/10.1109/TNNLS.2018.2875194.

    Google Scholar 

  • Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05) (vol. 1, pp. 886–893). IEEE.

    Google Scholar 

  • Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248–255). https://doi.org/10.1109/CVPR.2009.5206848.

  • Drucker, H., Burges, C. J., Kaufman, L., Smola, A., & Vapnik, V. (1996). Support vector regression machines. Advances in Neural Information Processing Systems, 9, 155–161.

    Google Scholar 

  • Dubey, R., Peterson, J., Khosla, A., Yang, M. H., & Ghanem, B. (2015). What makes an object memorable? In Proceedings of the ieee international conference on computer vision (pp. 1089–1097).

    Google Scholar 

  • Fried, O., Lu, J., Zhang, J., Měch, R., Echevarria, J., Hanrahan, P., & Landay, J. A. (2020). Adaptive photographic composition guidance. In Proceedings of the 2020 CHI conference on human factors in computing systems, CHI ’20 (pp. 1–13). ACM. https://doi.org/10.1145/3313831.3376635. https://dl.acm.org/doi/10.1145/3313831.3376635.

  • Engilberge, M., Chevallier, L., Perez, P., & Cord, M. (2019). Sodeep: A sorting deep net to learn ranking loss surrogates. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR).

    Google Scholar 

  • Everingham, M., & Winn, J. (2011). The pascal visual object classes challenge 2012 (voc2012) development kit. Pattern Analysis, Statistical Modelling and Computational Learning, Tech. Rep, vol. 8.

    Google Scholar 

  • Fajtl, J., Argyriou, V., Monekosso, D., & Remagnino, P. (2018). Amnet: Memorability estimation with attention. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6363–6372).

    Google Scholar 

  • Fan, R. E., Chang, K. W., Hsieh, C. J., Wang, X. R., & Lin, C. J. (2008). Liblinear: A library for large linear classification. Journal of Machine Learning Research, 9(Aug), 1871–1874.

    MATH  Google Scholar 

  • Farhadi, A., Endres, I., Hoiem, D., & Forsyth, D. (2009). Describing objects by their attributes. In 2009 IEEE conference on computer vision and pattern recognition (pp. 1778–1785). IEEE.

    Google Scholar 

  • Gabbard, J. L., Fitch, G. M., & Kim, H. (2014). Behind the glass: Driver challenges and opportunities for ar automotive applications. Proceedings of the IEEE, 102(2), 124–136.

    Google Scholar 

  • Goetschalckx, L., & Wagemans, J. (2019). MemCat: A new category-based image set quantified on memorability. PeerJ, 7, e8169. https://doi.org/10.7717/peerj.8169. https://peerj.com/articles/8169.

  • Goetschalckx, L., Moors, P., & Wagemans, J. (2018). Image memorability across longer time intervals. Memory, 26(5), 581–588. https://doi.org/10.1080/09658211.2017.1383435. https://doi.org/10.1080%2F09658211.2017.1383435.

  • Goetschalckx, L., Andonian, A., Oliva, A., & Isola, P. (2019a). Ganalyze: Toward visual definitions of cognitive image properties. In Proceedings of the IEEE international conference on computer vision (pp. 5744–5753).

    Google Scholar 

  • Goetschalckx, L., Moors, P., Vanmarcke, S., & Wagemans, J. (2019b). Get the picture? Goodness of image organization contributes to image memorability. Journal of Cognition, 2(1), 22.

    Google Scholar 

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, K. Q. Weinberger (Eds.) Advances in neural information processing systems (NIPS) (vol. 27, pp. 2672–2680). Red Hook, NY: Curran Associates.

    Google Scholar 

  • Gygli, M., Grabner, H., Riemenschneider, H., Nater, F., & Van Gool, L. (2013). The interestingness of images. In 2013 IEEE international conference on computer vision (ICCV). Red Hook, NY: IEEE. https://doi.org/10.1109/iccv.2013.205.

  • Han, J., Chen, C., Shao, L., Xintao, H., Jungong, H., & Tianming, L. (2015). Learning computational models of video memorability from fMRI brain imaging. IEEE Transactions on Cybernetics, 45(8), 1692–1703.

    Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).

    Google Scholar 

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9, 1735–80. https://doi.org/10.1162/neco.1997.9.8.1735.

    Google Scholar 

  • Hollingworth, A., & Henderson, J. M. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception and Performance, 28(1), 113.

    Google Scholar 

  • Hollingworth, A., Williams, C. C., & Henderson, J. M. (2001). To see and remember: Visually specific information is retained in memory from previously attended objects in natural scenes. Psychonomic Bulletin & Review, 8(4), 761–768.

    Google Scholar 

  • Holstein, K., Hong, G., Tegene, M., McLaren, B. M., & Aleven, V. (2018). The classroom as a dashboard: Co-designing wearable cognitive augmentation for k-12 teachers. In Proceedings of the 8th international conference on learning analytics and knowledge, LAK ’18 (pp. 79–88). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3170358.3170377.

    Google Scholar 

  • Huiskes, M. J., & Lew, M. S. (2008). The mir flickr retrieval evaluation. In Proceedings of the 1st ACM international conference on Multimedia information retrieval (pp. 39–43).

    Google Scholar 

  • Hung, W. L., Lee, Y. J., & Huang, P. H. (2016). Creative experiences, memorability and revisit intention in creative tourism. Current Issues in Tourism, 19(8), 763–770. https://doi.org/10.1080/13683500.2013.877422.

    Google Scholar 

  • Isola, P., Parikh, D., Torralba, A., & Oliva, A. (2011a). Understanding the intrinsic memorability of images. In Advances in neural information processing systems (pp. 2429–2437).

    Google Scholar 

  • Isola, P., Xiao, J., Torralba, A., & Oliva, A. (2011b). What makes an image memorable? In CVPR 2011 (pp. 145–152). IEEE.

    Google Scholar 

  • Isola, P., Xiao, J., Parikh, D., Torralba, A., & Oliva, A. (2014). What makes a photograph memorable? IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7), 1469–1482. https://doi.org/10.1109/tpami.2013.200. https://doi.org/10.1109%2Ftpami.2013.200.

  • Jaegle, A., Mehrpour, V., Mohsenzadeh, Y., Meyer, T., Oliva, A., & Rust, N. (2019). Population response magnitude variation in inferotemporal cortex predicts image memorability. eLife, 8. https://doi.org/10.7554/elife.47596.

  • Jahanian, A., Chai, L., & Isola, P. (2020). On the “steerability” of generative adversarial networks. In International conference on learning representations.

    Google Scholar 

  • Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014). Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia (pp. 675–678).

    Google Scholar 

  • Jing, P., Su, Y., Nie, L., & Gu, H. (2016). Predicting image memorability through adaptive transfer learning from external sources. IEEE Transactions on Multimedia, 19(5), 1050–1062.

    Google Scholar 

  • Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009). Learning to predict where humans look. In 2009 IEEE 12th international conference on computer vision (pp. 2106–2113). IEEE.

    Google Scholar 

  • Khosla, A., Xiao, J., Isola, P., Torralba, A., & Oliva, A. (2012a). Image memorability and visual inception. In SIGGRAPH Asia 2012 technical briefs (pp. 1–4).

    Google Scholar 

  • Khosla, A., Xiao, J., Torralba, A., & Oliva, A. (2012b). Memorability of image regions. In Advances in neural information processing systems (pp. 296–304).

    Google Scholar 

  • Khosla, A., Bainbridge, W. A., Torralba, A., & Oliva, A. (2013). Modifying the memorability of face photographs. In Proceedings of the IEEE international conference on computer vision (pp. 3200–3207).

    Google Scholar 

  • Khosla, A., Das Sarma, A., & Hamid, R. (2014). What makes an image popular? In Proceedings of the 23rd international conference on World wide web (pp. 867–876).

    Google Scholar 

  • Khosla, A., Raju, A. S., Torralba, A., & Oliva, A. (2015). Understanding and predicting image memorability at a large scale. In Proceedings of the IEEE international conference on computer vision (pp. 2390–2398).

    Google Scholar 

  • Kim, J., Yoon, S., & Pavlovic, V. (2013). Relative spatial features for image memorability. In Proceedings of the 21st ACM international conference on Multimedia (pp. 761–764).

    Google Scholar 

  • Kleinlein, R., García-Faura, Á., Luna Jimenez, C., Montero, J. M., Díaz-de María, F., & Fernández-Martínez, F. (2019). Predicting image aesthetics for intelligent tourism information systems. Electronics, 8(6), 671.

    Google Scholar 

  • Kong, H. K., Liu, Z., & Karahalios, K. (2018). Frames and slants in titles of visualizations on controversial topics. In Proceedings of the 2018 CHI conference on human factors in computing systems (pp. 1–12).

    Google Scholar 

  • Kong, H. K., Liu, Z., & Karahalios, K. (2019). Trust and recall of information across varying degrees of title-visualization misalignment. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–13).

    Google Scholar 

  • Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010a). Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. Journal of Experimental Psychology: General, 139(3), 558.

    Google Scholar 

  • Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010b). Scene memory is more detailed than you think: The role of categories in visual long-term memory. Psychological Science, 21(11), 1551–1556.

    Google Scholar 

  • Krafka, K., Khosla, A., Kellnhofer, P., Kannan, H., Bhandarkar, S., Matusik, W., & Torralba, A. (2016). Eye tracking for everyone. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2176–2184).

    Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Proceedings of the 25th international conference on neural information processing systems - Volume 1, NIPS’12 (pp. 1097–1105). Red Hook, NY, USA: Curran Associates Inc.

    Google Scholar 

  • Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J. R. R., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Duerig, T., & Ferrari, V. (2018). The open images dataset V4: Unified image classification, object detection, and visual relationship detection at scale. CoRR.206137 http://arxiv.org/abs/1811.00982.

  • Kyle-Davidson, C., Bors, A., & Evans, K. (2019). Predicting visual memory schemas with variational autoencoders. In BMVC 2019.

    Google Scholar 

  • Kyle-Davidson, C., Bors, A. G., & Evans, K. K. (2020). Generating memorable images based on human visual memory schemas. arXiv e-prints.

    Google Scholar 

  • Lazebnik, S., Schmid, C., & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06) (vol. 2, pp. 2169–2178). IEEE.

    Google Scholar 

  • Li, Y., Hou, X., Koch, C., Rehg, J. M., & Yuille, A. L. (2014). The secrets of salient object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 280–287).

    Google Scholar 

  • Li, X., Yi, W., Chi, H. L., Wang, X., & Chan, A. P. (2018). A critical review of virtual and augmented reality (vr/ar) applications in construction safety. Automation in Construction, 86, 150–162.

    Google Scholar 

  • Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common objects in context. In D. Fleet, T. Pajdla, B. Schiele, T. Tuytelaars (Eds.) 2014 European conference on computer vision (ECCV) (pp. 740–755). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-10602-1_48. https://doi.org/10.1007%2F978-3-319-10602-1_48.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Google Scholar 

  • Lu, J., Xu, M., Yang, R., & Wang, Z. (2018). What makes natural scene memorable? In Proceedings of the 2018 workshop on understanding subjective attributes of data, with the focus on evoked emotions (pp. 9–15)

    Google Scholar 

  • Lu, J., Xu, M., Yang, R., & Wang, Z. (2020). Understanding and predicting the memorability of outdoor natural scenes. IEEE Transactions on Image Processing, 29, 4927–4941. https://doi.org/10.1109/TIP.2020.2975957.

    Google Scholar 

  • Lukavský, J., & Děchtěrenko, F. (2017). Visual properties and memorising scenes: Effects of image-space sparseness and uniformity. Attention, Perception, & Psychophysics, 79(7), 2044–2054. https://doi.org/10.3758/s13414-017-1375-9. https://doi.org/10.3758%2Fs13414-017-1375-9.

  • Ma, S., Wei, Z., Tian, F., Fan, X., Zhang, J., Shen, X., Lin, Z., Huang, J., Měch, R., Samaras, D., et al. (2019). Smarteye: Assisting instant photo taking via integrating user preference with deep view proposal network. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–12).

    Google Scholar 

  • Machajdik, J., & Hanbury, A. (2010). Affective image classification using features inspired by psychology and art theory. In Proceedings of the 18th ACM international conference on Multimedia (pp. 83–92).

    Google Scholar 

  • Madan, C. R. (2020). Exploring word memorability: How well do different word properties explain item free-recall probability? Psychonomic Bulletin & Review, 1–13.

    Google Scholar 

  • Mahowald, K., Isola, P., Fedorenko, E., Gibson, E., & Oliva, A. (2018). Memorable words are monogamous: The role of synonymy and homonymy in word recognition memory. PsyArXiv.

    Google Scholar 

  • Mancas, M., & Le Meur, O. (2013). Memorability of natural scenes: The role of attention. In 2013 IEEE international conference on image processing (pp. 196–200). IEEE.

    Google Scholar 

  • Miller, G. A. (1995). Wordnet: a lexical database for english. Communications of the ACM, 38(11), 39–41.

    Google Scholar 

  • Monfort, M., Andonian, A., Zhou, B., Ramakrishnan, K., Bargal, S. A., Yan, Y., Brown, L., Fan, Q., Gutfreund, D., Vondrick, C., et al. (2019). Moments in time dataset: one million videos for event understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2), 502–508.

    Google Scholar 

  • Murray, N., Marchesotti, L., & Perronnin, F. (2012). Ava: A large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition (pp. 2408–2415). IEEE.

    Google Scholar 

  • Newman, A., Bylinskii, Z., Haroz, S., Madan, S., Durand, F., & Oliva, A. (2018). Effects of title wording on memory of trends in line graphs. Journal of Vision, 18(10), 837.

    Google Scholar 

  • Newman, A., Fosco, C., Casser, V., Lee, A., Barry, Mcnamara, & Oliva, A. (2020). Multimodal memorability: Modeling effects of semantics and decay on video memorability. In ECCV.

    Google Scholar 

  • Oliva, A., & Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42(3), 145–175.

    MATH  Google Scholar 

  • Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., & Hays, J. (2016). Webgazer: Scalable webcam eye tracking using user interactions. In Proceedings of the twenty-fifth international joint conference on artificial intelligence-IJCAI 2016.

    Google Scholar 

  • Park, S., Aksan, E., Zhang, X., & Hilliges, O. (2020). Towards end-to-end video-based eye-tracking. In European conference on computer vision (pp. 747–763). Springer.

    Google Scholar 

  • Perera, S., Tal, A., & Zelnik-Manor, L. (2019). Is image memorability prediction solved? In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 0–0).

    Google Scholar 

  • Qian, J., Chapin, A., Papoutsaki, A., Yang, F., Nelissen, K., & Huang, J. (2018). Remotion: A motion-based capture and replay platform of mobile device interaction for remote usability testing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(2), 1–18.

    Google Scholar 

  • Ramanathan, S., Katti, H., Sebe, N., Kankanhalli, M., & Chua, T. S. (2010). An eye fixation database for saliency detection in images. In European conference on computer vision (pp. 30–43). Springer.

    Google Scholar 

  • Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015). Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211–252.

    MathSciNet  Google Scholar 

  • Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008). Labelme: a database and web-based tool for image annotation. International Journal of Computer Vision, 77(1-3), 157–173.

    Google Scholar 

  • Rust, N. C., & Mehrpour, V. (2020). Understanding image memorability. Trends in Cognitive Sciences, 24(7), 557–568. https://doi.org/https://doi.org/10.1016/j.tics.2020.04.001. https://www.sciencedirect.com/science/article/pii/S1364661320301030.

  • Saleh, B., Farhadi, A., & Elgammal, A. (2013). Object-centric anomaly detection by attribute-based reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 787–794).

    Google Scholar 

  • Sharif Razavian, A., Azizpour, H., Sullivan, J., & Carlsson, S. (2014). Cnn features off-the-shelf: an astounding baseline for recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 806–813).

    Google Scholar 

  • Shekhar, S., Singal, D., Singh, H., Kedia, M., & Shetty, A. (2017). Show and recall: Learning what makes videos memorable. In Proceedings of the IEEE international conference on computer vision (pp. 2730–2739).

    Google Scholar 

  • Siarohin, A., Zen, G., Majtanovic, C., Alameda-Pineda, X., Ricci, E., & Sebe, N. (2017). How to make an image more memorable? A deep style transfer approach. In Proceedings of the 2017 ACM on international conference on multimedia retrieval (pp. 322–329).

    Google Scholar 

  • Siarohin, A., Zen, G., Majtanovic, C., Alameda-Pineda, X., Ricci, E., & Sebe, N. (2019). Increasing image memorability with neural style transfer. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 15(2), 1–22.

    Google Scholar 

  • Sidorov, O. (2019). Changing the image memorability: From basic photo editing to gans. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) workshops.

    Google Scholar 

  • Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 1904–1920, 3(3), 271–295. https://doi.org/10.1111/j.2044-8295.1910.tb00206.x.

    Google Scholar 

  • Standing, L. (1973). Learning 10000 pictures. The Quarterly Journal of Experimental Psychology, 25(2), 207–222.

    Google Scholar 

  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9).

    Google Scholar 

  • Võ, M. L. H., Bylinskii, Z., & Oliva, A. (2017). Image memorability in the eye of the beholder: Tracking the decay of visual scene representations. bioRxiv. https://doi.org/10.1101/141044. https://www.biorxiv.org/content/early/2017/05/24/141044.

  • Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010). Locality-constrained linear coding for image classification. In 2010 IEEE computer society conference on computer vision and pattern recognition (pp. 3360–3367). IEEE.

    Google Scholar 

  • Wolfe, J. M., Horowitz, T. S., & Michod, K. O. (2007). Is visual attention required for robust picture memory? Vision Research, 47(7), 955–964.

    Google Scholar 

  • Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010). Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition (pp. 3485–3492). IEEE.

    Google Scholar 

  • Xiong, C., Ceja, C. R., Ludwig, C. J., & Franconeri, S. (2019). Biased average position estimates in line and bar graphs: Underestimation, overestimation, and perceptual pull. IEEE Transactions on Visualization and Computer Graphics, 26(1), 301–310.

    Google Scholar 

  • Zarezadeh, S., Rezaeian, M., & Sadeghi, M. T. (2017). Image memorability prediction using deep features. In 2017 Iranian conference on electrical engineering (ICEE) (pp. 2176–2181). IEEE.

    Google Scholar 

  • Zhang, X., Sugano, Y., Fritz, M., & Bulling, A. (2015). Appearance-based gaze estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

    Google Scholar 

  • Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., & Oliva, A. (2014). Learning deep features for scene recognition using places database. Advances in Neural Information Processing Systems, 27, 487–495.

    Google Scholar 

  • Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2017). Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6), 1452–1464.

    Google Scholar 

  • Zhu, J., Shen, Y., Zhao, D., & Zhou, B. (2020). In-domain GAN inversion for real image editing. In European conference on computer vision (ECCV).

    Google Scholar 

Download references

Acknowledgements

We thank the Vannevar Bush Faculty Fellowship Program of the ONR (N00014-16-1-3116 to A.O.). Thank you also to Wilma A. Bainbridge for her insightful comments on prior versions of this chapter, as well as to the other editors of this collection.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zoya Bylinskii .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Bylinskii, Z., Goetschalckx, L., Newman, A., Oliva, A. (2022). Memorability: An Image-Computable Measure of Information Utility. In: Ionescu, B., Bainbridge, W.A., Murray, N. (eds) Human Perception of Visual Information. Springer, Cham. https://doi.org/10.1007/978-3-030-81465-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-81465-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-81464-9

  • Online ISBN: 978-3-030-81465-6

  • eBook Packages: Computer ScienceComputer Science (R0)