Skip to main content

It’s too much for us to handle—The effect of smartphone use on long-term retrieval of family photos

Personal and Ubiquitous Computing (2022)Cite this article

Abstract

We tested the use of smartphones for retrieval of pictures of long-term, salient family events. Our goal was to replicate a study conducted a decade ago where participants used digital cameras. We found that smartphones affected picture retrieval in two contrasting ways. Overall, the constant availability of smartphones increased collection size. This increased the failure percentage for pictures downloaded to computer file system folders from an average of 43% in the original study to 71% in the current one. On the other hand, improved smartphone retrieval technologies including timeline, search, and face recognition reduced smartphone application retrieval failures to 29% on average. Overall, these two opposing tendencies canceled each other out, with no significant difference in failure percentage and retrieval time between the two studies. Results indicate that the magnitude of pictures is too much for us to manually handle and we must rely on technology for picture retrieval.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Data availability

N/A.

Code availability

N/A.

References

  1. Aris A, Gemmell J, Lueder R (2004) Exploiting location and time for photo search and storytelling in MyLifeBits. Technical Report MSR-TR-2004-102

  2. Bao X, Herlocker JL, Dietterich T (2006) Fewer clicks and less frustration: reducing the cost of reaching the right folder. Paper presented at the Proceedings of the 11th international conference on Intelligent user interfaces. ACM, New York, pp 178–185

  3. Barasch A, Diehl K, Silverman J, Zauberman G (2017) Photographic memory: the effects of volitional photo taking on memory for visual and auditory aspects of an experience. Psychol Sci 28(8):1056–1066

    Article  Google Scholar 

  4. Barreau DK, Nardi BA (1995) Finding and reminding: file organization from the desktop. SIGCHI Bulletin 27(3):39–43

    Article  Google Scholar 

  5. Benn Y, Bergman O, Glazer L, Arent P, Wilkinson ID, Varley R, Whittaker S (2015) Navigating through digital folders uses the same brain structures as real world navigation. Sci Rep 5:1–8

    Article  Google Scholar 

  6. Bergman O, Beyth-Marom R, Nachmias R, Gradovitch N, Whittaker S (2008) Improved search engines and navigation preference in personal information management. ACM Trans Inf Syst 26(4):1–24. https://doi.org/10.1145/1402256.1402259

    Article  Google Scholar 

  7. Bergman O, Israeli T, Whittaker S (2019) Search is the future? The young search less for files. Proc Assoc Inf Sci Technol 56(1):360–363

    Article  Google Scholar 

  8. Bergman O, Israeli T, Whittaker S (2020) Factors hindering shared files retrieval. Aslib J Inf Manag 72(1):130–147

    Article  Google Scholar 

  9. Bergman O, Israeli T, Whittaker S (2020) The scalability of different file sharing methods. J Am Soc Inf Sci 71(12):1424–1438. https://doi.org/10.1002/asi.24350

    Article  Google Scholar 

  10. Bergman O, Tene-Rubinstein M, Shalom J (2013) The use of attention resources in navigation vs search. Pers Ubiquit Comput 17(3):583–590

    Article  Google Scholar 

  11. Bergman O, Tucker S, Beyth-Marom R, Cutrell E, Whittaker S (2009) It’s not that important: demoting personal information of low subjective importance using GrayArea. CHI 2009 Conference on Human Factors and Computing Systems. ACM, New York, NY, USA, pp 269–278

    Google Scholar 

  12. Bergman O, Tucker S, Dahamshy S (2018) The effect of demoting near-duplicate pictures. Proc Assoc Info Sci Technol 55(1):755–756

    Article  Google Scholar 

  13. Bergman O, Whittaker S (2016) The science of managing our digital stuff. Mit Press, Cambridge, MA

    Book  Google Scholar 

  14. Bergman O, Whittaker S, Falk N (2014) Shared files – the retrieval perspective. J Am Soc Inform Sci Technol 65(10):1949–1963. https://doi.org/10.1002/asi.23147

    Article  Google Scholar 

  15. Bergman O, Whittaker S, Sanderson M, Nachmias R, Ramamoorthy A (2010) The effect of folder structure on personal file navigation. J Am Soc Inform Sci Technol 61(12):2426–2441

    Article  Google Scholar 

  16. Bergman O, Whittaker S, Schooler J (2021) Out of sight and out of mind: bookmarks are created but not used. J Librariansh Inf Sci 53(2):338–348

    Article  Google Scholar 

  17. Bergman O, Yanai N (2018) Personal information retrieval: smartphones vs. computers, emails vs. files. Pers Ubiquit Comput 22(4):621–632

    Article  Google Scholar 

  18. Berlin LM, Jeffries R, O’Day VL, Paepcke A, Wharton C (1993) Where did you put it? Issues in the design and use of a group memory. Paper presented at the Proceedings of the INTERACT’93 and CHI’93 conference on Human factors in computing systems, New York. ACM, New York, pp 23–30

  19. Boardman R, Sasse MA (2004) “Stuff goes into the computer and doesn’t come out”: a cross-tool study of personal information management Paper presented at the SIGCHI conference on Human Factors in Computing Systems, Vienna, Austria. ACM, New York, pp 583–590

  20. Brehmer M, Lee B, Bach B, Riche NH, Munzner T (2016) Timelines revisited: a design space and considerations for expressive storytelling. IEEE Trans Visual Comput Graphics 23(9):2151–2164

    Article  Google Scholar 

  21. Capra RG, Pérez-Quiñones MA (2005) Using web search engines to find and refind information. Computer 38(10):36–42

    Article  Google Scholar 

  22. Chen AYS, Odom W, Zhong C, Lin H, Amram T (2019) Chronoscope: designing temporally diverse interactions with personal digital photo collections. Paper presented at the Proceedings of the 2019 on Designing Interactive Systems Conference. ACM, New York, pp 799–812

  23. Crowder JW, Marion JS, Reilly M (2015) File naming in digital media research: examples from the humanities and social sciences. J Librariansh Sch Commun 3(3):eP1260

  24. Duel T, Frohlich DM, Kroos C, Xu Y, Jackson PJ, and Plumbley MD (2018) Supporting audiography: design of a system for sentimental sound recording, classification and playback. Paper presented Int Conf Hum-Comput Interaction

  25. Durrant A, Frohlich D, Sellen A, Lyons E (2009) Home curation versus teenage photography: photo displays in the family home. Int J Hum Comput Stud 67(12):1005–1023

    Article  Google Scholar 

  26. Elsweiler D, Baillie M, Ruthven I (2008) Exploring memory in email refinding. ACM Trans Inf Syst 26(4):1–36. https://doi.org/10.1145/1402256.1402260

    Article  Google Scholar 

  27. Fitchett S (2014) Understanding and improving personal file retrieval. (Ph.D.). Canterbury, Canterbury, Australia

  28. Fitchett S, Cockburn A (2015) An empirical characterisation of file retrieval. Int J Hum Comput Stud 74:1–13

    Article  Google Scholar 

  29. Frohlich DM, Wall S, Kiddle G (2013) Rediscovery of forgotten images in domestic photo collections. Pers Ubiquit Comput 17(4):729–740

    Article  Google Scholar 

  30. Golsteijn C, Van Den Hoven E, Frohlich D, and Sellen A (2012) Towards a more cherishable digital object. Paper presented at the Proceedings of the designing interactive systems conference

  31. Güldenpfennig F, Fitzpatrick G (2015) Personal digital archives on mobile phones with MEO. Pers Ubiquit Comput 19(2):445–461

    Article  Google Scholar 

  32. Güldenpfennig F, Ganhör R, and Fitzpatrick G (2017) How to look at two-sided photos? exploring novel perspectives on digital images. Paper presented at the Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

  33. Hermans L, Broekhuijsen M, and Markopoulos P (2017) Memora: a design for teenagers to connect virtual and physical possessions. Paper presented at the Proceedings of the European Conference on Cognitive Ergonomics 2017

  34. Jan S, Li M, Al-Raweshidy H (2011) FARM: file annotation and retrieval on mobile devices. Pers Ubiquit Comput 15(7):771–779

    Article  Google Scholar 

  35. Jansen M, van den Hoven E, Frohlich D (2014) Pearl: living media enabled by interactive photo projection. Pers Ubiquit Comput 18(5):1259–1275

    Google Scholar 

  36. Jensen C, Lonsdale H, Wynn E, Cao J, Slater M, and Dietterich TG (2010) The life and times of files and information: a study of desktop provenance. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Atlanta, Georgia, USA

  37. Jiang L, Kalantidis Y, Cao L, Farfade S, Tang J, and Hauptmann A G (2017) Delving deep into personal photo and video search. Paper presented at the Proceedings of the Tenth ACM International Conference on Web Search and Data Mining

  38. Jones E, Bruce H, Klasnja P, Jones W (2008) I give up! Five factors that contribute to the abandonment of information management strategies. Proc Am Soc Inf Sci Technol 45(1):1–6

    Google Scholar 

  39. Karlsson K, Jiang W, and Zhang DQ (2014) Mobile photo album management with multiscale timeline. Paper presented Proc 22nd ACM Int Conf Multimedia

  40. Kirk D, Sellen A, Rother C, Wood K (2006) Understanding photowork. SIGCHI conference on Human Factors in Computing Systems. ACM, New York, pp 761–770

    Chapter  Google Scholar 

  41. Kislinger L, Kotrschal K (2021) Hunters and gatherers of pictures: why photography has become a human universal. Front Psychol 12:1–21

    Article  Google Scholar 

  42. Kumar N, Seitz S (2014) Photo recall: using the internet to label your photos. Paper presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 771–778

  43. Lee J (2020) Algorithmic uses of cybernetic memory: Google photos and a genealogy of algorithmically generated “memory.” Soc Media+ Soc 6(4):2056305120978968

    Google Scholar 

  44. Lonn S, Radeva P, Dimiccoli M (2019) Smartphone picture organization: a hierarchical approach. Comput Vision and Image Underst 187:102789

    Article  Google Scholar 

  45. Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM (2011) mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof Psychol Res Pract 42(6):505

    Article  Google Scholar 

  46. McGookin D (2019) Reveal: investigating proactive location-based reminiscing with personal digital photo repositories. Paper presented at the Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–14

  47. Narayanan A, Ramadan E, Carpenter J, Liu Q, Liu Y, Qian F, and Zhang ZL (2020) A first look at commercial 5G performance on smartphones. Paper presented at the Proceedings of The Web Conference. ACM, New York, pp 894–905

  48. Nasar MRA, Mohd M, Ali NM (2018) Personal information management: evaluation of the photo refinder system. Adv Sci Lett 24(2):1090–1094

    Article  Google Scholar 

  49. Neisser U (1964) Visual search. Sci Am 210(6):94–102

    Article  Google Scholar 

  50. Obotivere A, Nwaezeigwe A (2020) The debut of 5G wireless technology. Int J Pure Appl Sci 19(9):1–20

    Google Scholar 

  51. Peesapati ST, Schwanda V, Schultz J, Lepage M, Jeong SY, Cosley D (2010) Pensieve: supporting everyday reminiscence. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, pp 2027–2036

  52. Petrelli D, Bowen S, Whittaker S (2014) Photo mementos: designing digital media to represent ourselves at home. Int J Hum Comput Stud 72(3):320–336

    Article  Google Scholar 

  53. Savchenko A (2020) Event recognition with automatic album detection based on sequential grouping of confidence scores and neural attention. Paper presented at the 2020 International Joint Conference on Neural Networks (IJCNN), pp 1–8

  54. Shah R, Kwatra V (2012) All smiles: automatic photo enhancement by facial expression analysis. Paper presented at the Proceedings of the 9th European Conference on Visual Media Production, pp 1–10

  55. Soares JS, Storm BC (2018) Forget in a flash: a further investigation of the photo-taking-impairment effect. J Appl Res Mem Cogn 7(1):154–160

    Article  Google Scholar 

  56. Teevan J, Alvarado C, Ackerman MS, Karger DR (2004) The perfect search engine is not enough: a study of orienteering behavior in directed search. In: Dykstra-Erickson E, Tscheligi M (eds) SIGCHI conference on Human Factors in Computing Systems. ACM Press, New York, pp 415–422

    Chapter  Google Scholar 

  57. Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12:97–136

    Article  Google Scholar 

  58. Tsai WC, Chen AYS, Hsu SY, Liang RH (2015) CrescendoMessage: interacting with slow messaging. Paper presented at the Proceedings of the International Association of Societies of Design Research Conference. ACM, New York, pp 2078–2095

  59. Wang D, Otto C, Jain AK (2016) Face search at scale. IEEE Trans Pattern Anal Mach Intell 39(6):1122–1136

    Article  Google Scholar 

  60. Whittaker S, Bergman O, Clough P (2010) Easy on that trigger dad: a study of long term family photo retrieval. Pers Ubiquit Comput 13(5):17–30

    Google Scholar 

  61. Whittaker S, Matthews T, Cerruti J, Badenes H, Tang J (2011) Am I wasting my time organizing email? A study of email refinding. In: Tan D, Fitzpatrick G, Gutwin C, Begole B, Kellogg WA (eds) Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, pp 3449-3458

  62. Williamson J, Brown LM (2008) Flutter: directed random browsing of photo collections with a tangible interface. Paper presented at the Proceedings of the 7th ACM conference on Designing interactive systems. ACM, New York, pp 147–155

  63. Yates DB (2021) Development and implementation of locally-executed data mining on smartphones. Doctoral dissertation, Charles Sturt University Australia 8

  64. Zürn X, Broekhuijsen M, van Gennip D, Bakker S, Zijlema A, Van Den Hoven E (2019) Stimulating photo curation on smartphones. Paper presented at the Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, pp 255–259

  65. Zürn X, Damen K, van Leiden F, Broekhuijsen M, and Markopoulos P (2017) Photo curation practices on smartphones. Paper presented at the International Conference on Advances in Computer Entertainment

Download references

Acknowledgements

We thank Prof. Paul Clough and our participants.

Funding

This research was funded by the Israeli Science Foundation, Grant 1074/16.

Author information

Authors and Affiliations

  1. Information Science Department, Bar-Ilan University, Ramat-Gan, Israel

    Ofer Bergman & Diana Gutman

  2. Jack Baskin School of Engineering, University of California, Santa Cruz, Santa Cruz, CA, USA

    Steve Whittaker

Authors
  1. Ofer Bergman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diana Gutman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steve Whittaker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ofer Bergman.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bergman, O., Gutman, D. & Whittaker, S. It’s too much for us to handle—The effect of smartphone use on long-term retrieval of family photos. Pers Ubiquit Comput (2022). https://doi.org/10.1007/s00779-022-01677-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00779-022-01677-x

Keywords

  • Photography
  • Long-term retrieval
  • Smartphones
  • Personal information management