Abstract
With the great success of convolutional neural networks (CNN) achieved on various computer vision tasks in recent years, CNN has also been applied in natural image saliency prediction. As a specific visual stimuli, webpages exhibit evident similarities whereas also significant differences from natural image. Consequently, the learned CNN for natural image saliency prediction cannot be directly used to predict webpage saliency. Only a few researches on webpage saliency prediction have been developed till now. In this paper, we propose a simple yet effective scheme of two-stage transfer learning of end-to-end CNN to predict the webpage saliency. In the first stage, the output layer of two typical CNN architectures with instances of AlexNet and VGGNet are reconstructed, and the parameters between the fully connected layers are relearned from a large natural image database for image saliency prediction. In the second stage, the parameters between the fully connected layers are relearned from a scarce webpage database for webpage saliency prediction. In fact, the two-stage transfer learning can be regarded as a task transfer in the first stage and a domain transfer in the second stage, respectively. The experimental results indicate that the proposed two-stage transfer learning of end-to-end CNN can obtain a substantial performance improvement for webpage saliency prediction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)
Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 185–207 (2013)
Vig, E., Dorr, M., Cox, D.: Large-scale optimization of hierarchical features for saliency prediction in natural images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2798–2805 (2014)
Kümmerer, M., Theis, L., Bethge, M.: Deep gaze I: boosting saliency prediction with feature maps trained on imagenet (2014). arXiv preprint arXiv:1411.1045
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Still, J.D., Masciocchi, C.M.: A saliency model predicts fixations in web interfaces. In: 5th International Workshop on Model Driven Development of Advanced User Interfaces, pp. 25–28 (2010)
Buscher, G., Cutrell, E., Morris, M.R.: What do you see when you’re surfing?: using eye tracking to predict salient regions of web pages. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 21–30. ACM (2009)
Nielsen, J.: F-shaped pattern for reading web content (2006). http://www.nngroup.com/articles/f-shaped-pattern-reading-web-content
Shen, C., Huang, X., Zhao, Q.: Predicting eye fixations on webpage with an ensemble of early features and high-level representations from deep network. IEEE Trans. Multimed. 17(11), 2084–2093 (2015)
Li, J., Su, L., Wu, B., et al.: Webpage saliency prediction with multi-features fusion. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 674–678 (2016)
Weiss, K., Khoshgoftaar, T.M., Wang, D.D.: A survey of transfer learning. J. Big Data 3(1), 1–40 (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv preprint arXiv:1409.1556
Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Jiang, M., Huang, S., Duan, J., et al.: SALICON: saliency in context. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1072–1080 (2015)
Shen, C., Zhao, Q.: Webpage saliency. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 33–46. Springer, Cham (2014). doi:10.1007/978-3-319-10584-0_3
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). doi:10.1007/978-3-319-10602-1_48
Garcia-Diaz, A., Leboran, V., Fdez-Vidal, X.R., et al.: On the relationship between optical variability, visual saliency, and eye fixations: a computational approach. J. Vis. 12(6), 17 (2012)
Acknowledgments
This work was supported by Shanghai Municipal Natural Science Foundation under Grant No. 16ZR1411100 and National Natural Science Foundation of China under Grant No. 61471230.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Shan, W., Sun, G., Zhou, X., Liu, Z. (2017). Two-Stage Transfer Learning of End-to-End Convolutional Neural Networks for Webpage Saliency Prediction. In: Sun, Y., Lu, H., Zhang, L., Yang, J., Huang, H. (eds) Intelligence Science and Big Data Engineering. IScIDE 2017. Lecture Notes in Computer Science(), vol 10559. Springer, Cham. https://doi.org/10.1007/978-3-319-67777-4_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-67777-4_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67776-7
Online ISBN: 978-3-319-67777-4
eBook Packages: Computer ScienceComputer Science (R0)