Collaborative Personalization of Image Enhancement

  • Ashish Kapoor
  • Juan C. Caicedo
  • Dani Lischinski
  • Sing Bing Kang
Article

Abstract

This paper presents methods for personalization of image enhancement, which could be deployed in photo editing software and also in cloud-based image sharing services. We observe that users do have different preferences for enhancing images and that there are groups of people that share similarities in preferences. Our goal is to predict enhancements for novel images belonging to a particular user based on her specific taste, to facilitate the retouching process on large image collections. To that end, we describe an enhancement framework that can learn user preferences in an individual or collaborative way. The proposed system is based on a novel interactive application that allows to collect user’s enhancement preferences. We propose algorithms to predict personalized enhancements by learning a preference model from the provided information. Furthermore, the algorithm improves prediction performance as more enhancement examples are progressively added. We conducted experiments via Amazon Mechanical Turk to collect preferences from a large group of people. Results show that the proposed framework can suggest image enhancements more targeted to individual users than commercial tools with global auto-enhancement functionalities.

Keywords

Image enhancement Personalization Collaborative filtering  Crowdsourcing 

References

  1. Buchsbaum, G. (1980). A spatial processor model for object colour perception. Journal of the Franklin Institute, 310, 337–350.CrossRefGoogle Scholar
  2. Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011). Learning photographic global tonal adjustment with a database of input/output image Pairs. In: CVPR’11.Google Scholar
  3. Caicedo, J., Kapoor, A., & Kang, S. B. (2011). Collaborative personalization of image enhancement. In CVPR’11.Google Scholar
  4. Chua, T. S., Tang, J., Hong, R., Li, H., Luo, Z., & Zheng, Y. (2009). Nus-wide: A real-world web image database from national university of singapore. In CIVR ’09, ACM.Google Scholar
  5. Dale, K., Johnson, M. K., Sunkavalli, K., Matusik, W., & Pfister, H. (2009). Image restoration using online photo collections. In ICCV.Google Scholar
  6. Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Processing, 54(12), 3736–3745.CrossRefMathSciNetGoogle Scholar
  7. Farid, H. (2001). Blind inverse gamma correction. IEEE Transactions on Image Processing, 10(10), 1428–1433.CrossRefMATHGoogle Scholar
  8. Farid, H., & Popescu, A. C. (2001). Blind removal of lens distortions. Journal of the Optical Society of America, 18(9), 2072–2078.CrossRefGoogle Scholar
  9. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3), 787–794.CrossRefGoogle Scholar
  10. Fogarty, J., Tan, D., Kapoor, A., & Winder, S. (2008). Cueflik: Interactive concept learning in image search. In Conference on human factors in computing systems (CHI).Google Scholar
  11. Freeman, W. T., Jones, T. R., & Pasztor, E. C. (2002). Example-based super-resolution. IEEE Computer Graphics and Applications, 22, 56–65.Google Scholar
  12. Gehler, P. V., Rother, C., Blake, A., Minka, T., & Sharp, T. (2008). Bayesian color constancy revisited. In CVPR.Google Scholar
  13. Gijsenij, A., & Gevers, T. (2007). Color constancy using natural image statistics. In CVPR.Google Scholar
  14. Grabler, F., Agrawala, M., Li, W., Dontcheva, M., & Igarashi, T. (2009). Generating photo manipulation tutorials by demonstration. ACM Transactions on Graphics, 28(3), 1–9.CrossRefGoogle Scholar
  15. Hays, J., & Efros, A. (2008). Scene completion using millions of photographs. Communications of the ACM, 51(10), 87–94.CrossRefGoogle Scholar
  16. Hsu, E., Mertens, T., Paris, S., Avidan, S., & Durand, F. (2008). Light mixture estimation for spatially varying white balance. ACM Transactions on Graphics and SIGGRAPH, 27(3), article 70.Google Scholar
  17. Jain, P., Kulis, B., Dhillon, I., & Grauman, K. (2008). Online metric learning and fast similarity search. In NIPS.Google Scholar
  18. Joshi, N., Matusik, W., Adelson, E. H., & Kriegman, D. J. (2010). Personal photo enhancement using example images. ACM Transactions on Graphics, 29(2), 1–15.CrossRefGoogle Scholar
  19. Kang, S. B. (2007). Automatic removal of chromatic aberration from a single image. In CVPR.Google Scholar
  20. Kang, S. B., Kapoor, A., & Lischinski, D. (2010). Personalization of image enhancement. In CVPR.Google Scholar
  21. Kapoor, A., Ahn, H., & Picard, R. W. (2005). Mixture of Gaussian processes for combining multiple modalities. In Workshop on multiple classifier systems, 2005.Google Scholar
  22. Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. IEEE, Computer Journal, 42(8), 30–37.CrossRefGoogle Scholar
  23. Krause, A., Singh, A., & Guestrin, C. (2008). Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies. Journal of Machine Learning Research, 9, 235–284.MATHGoogle Scholar
  24. Lawrence, N. D., & Urtasun R. (2009). Non-linear matrix factorization with Gaussian processes. In Proceedings of the 26th annual international conference on machine learning. ACM, 2009.Google Scholar
  25. Lin, S., Gu, J., Yamazaki, S., & Shum, H. Y. (2004). Radiometric calibration using a single image. In CVPR (Vol. 2, pp. 938–945).Google Scholar
  26. Liu, D. C., & Nocedal, J. (1989). On the limited memory method for large scale optimization. Mathematical Programming B, 45(3), 503–528.Google Scholar
  27. Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009). Non-local sparse models for image restoration. In ICCV.Google Scholar
  28. Marks, J., Andalman, B., Beardsley, P., Freeman, W., Gibson, S., Hodgins, J., et al. (1997). Design galleries: A general approach to setting parameters for computer graphics and animation. In ACM SIGGRAPH (pp. 389–400).Google Scholar
  29. Portilla, J., Strela, V., Wainwright, M. J., & Simoncelli, E. P. (2003). Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12(11), 1338–1351.CrossRefMATHMathSciNetGoogle Scholar
  30. Rennie, J. D. M., & Nathan, S. (2005). Fast maximum margin matrix factorization for collaborative prediction. In Proceedings of the 22nd international conference on machine learning. ACM, 2005.Google Scholar
  31. Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008). Labelme: A database and web-based tool for image annotation. IJCV, 77(1–3), 157–173.CrossRefGoogle Scholar
  32. Salakhutdinov, R., & Mnih, A. (2007). Probabilistic matrix factorization. In Advances in neural information processing systems (Vol. 20). MIT, Cambridge, MA.Google Scholar
  33. Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based collaborative filtering recommendation algorithms. In WWW. New York, NY: ACM.Google Scholar
  34. Seeger, M. (2004). Gaussian processes for machine learning. International Journal of Neural Systems, 14(2), 69–106.CrossRefGoogle Scholar
  35. Shapira, L., Shamir, A., & Cohen-Or, D. (2009). Image appearance exploration by model based navigation.Google Scholar
  36. Stanikunas, R. (2004). Investigation of color constancy with a neural network. Neural Networks, 17(3), 327–337. Google Scholar
  37. Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2319–2323.Google Scholar
  38. Tresp, V. (2001). Neural Information Processing Systems: Mixture of Gaussian Processes 2001.Google Scholar
  39. Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics, 1, 80–83.CrossRefGoogle Scholar
  40. Wyszecki, G., & Stiles, W. S. (1982). Color science: Concepts and methods quantitative data and formulae. New York: Wiley.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Ashish Kapoor
    • 1
  • Juan C. Caicedo
    • 2
  • Dani Lischinski
    • 3
  • Sing Bing Kang
    • 1
  1. 1.Microsoft ResearchRedmondUSA
  2. 2.University of Illinois at Urbana-ChampaignUrbanaUSA
  3. 3.The Hebrew University of JerusalemJerusalem Israel

Personalised recommendations