Skip to main content

Where is my puppy? Retrieving lost dogs by facial features

Multimedia Tools and Applications volume 76, pages 15325–15340 (2017)Cite this article

Abstract

A pet that goes missing is among many people’s worst fears: a moment of distraction is enough for a dog or a cat wandering off from home. Some measures help matching lost animals to their owners; but automated visual recognition is one that — although convenient, highly available, and low-cost — is surprisingly overlooked. In this paper, we inaugurate that promising avenue by pursuing face recognition for dogs. We contrast four ready-to-use human facial recognizers (EigenFaces, FisherFaces, LBPH, and a Sparse method) to two original solutions based upon convolutional neural networks: BARK (inspired in architecture-optimized networks employed for human facial recognition) and WOOF (based upon off-the-shelf OverFeat features). Human facial recognizers perform poorly for dogs (up to 60.5 % accuracy), showing that dog facial recognition is not a trivial extension of human facial recognition. The convolutional network solutions work much better, with BARK attaining up to 81.1 % accuracy, and WOOF, 89.4 %. The tests were conducted in two datasets: Flickr-dog, with 42 dogs of two breeds (pugs and huskies); and Snoopybook, with 18 mongrel dogs.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. http://www.humanesociety.org/issues/pet_overpopulation/facts/pet_ownership_statistics.html(December 2015).

  2. https://www.aspca.org/about-us/press-releases/how-many-pets-are-lost-how-many-find-their-way-home-aspca-survey-has-answers https://www.aspca.org/about-us/press-releases/how-many-pets-are-lost-how-many-find-their-way-home-aspca-survey-has-answers(March 2015).

  3. For example, in August 2015, the New England Aquarium and MathWorks have launched a large competition for identifying individual endangered whales: https://www.kaggle.com/c/noaa-right-whale-recognition.

  4. http://www.robots.ox.ac.uk/~vgg/data/pets/

  5. http://cilvr.nyu.edu/doku.php?id=software:overfeat:start (August 2015).

  6. Available at:http://www.recod.ic.unicamp.br/~rwerneck/datasets/flickr-dog/.

  7. www.github.com/giovanichiachia/simple-hp.

References

  1. Ahonen T, Hadid A, Pietikinen M (2004) Face recognition with local binary patterns. In: Pajdla T, Matas J (eds) European conference on computer vision, lecture notes in computer science. doi:10.1007/978-3-540-24670-1_36, vol 3021. Springer, Berlin Heidelberg, pp 469–481

  2. Ahonen T, Hadid A, Pietikäinen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28:2037–2041. doi:10.1109/TPAMI.2006.244

    Article  MATH  Google Scholar 

  3. Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720. doi:10.1109/34.598228

    Article  Google Scholar 

  4. Belhumeur P N, Jacobs D W, Kriegman D J, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. In: IEEE transactions on pattern analysis and machine intelligence, vol 35, pp 2930– 2940

  5. Bergstra J, Yamins D, Cox D (2013) Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: International conference on machine learning. http://jmlr.org/proceedings/papers/v28/bergstra13.html, pp 115–123

  6. Bradski G (2000) The OpenCV library. Dr Dobb’s Journal of Software Tools

  7. Chiachia G, Falcão AX, Pinto N, Rocha A, Cox D (2014) Learning person-specific representations from faces in the wild. IEEE Trans Inf Forensics Secur 9(12):2089–2099. doi:10.1109/TIFS.2014.2359543. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6905816&tag=1

  8. Cox D, Pinto N (2011) Beyond simple features: a large-scale feature search approach to unconstrained face recognition. In: IEEE International conference on automatic face and gesture recognition and workshops. doi:10.1109/FG.2011.5771385, pp 8–15

  9. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE conference on computer vision and pattern recognition. doi:10.1109/CVPR.2005.177, vol 1, pp 886–893

  10. Diamond R, Carey S (1986) Why faces are and are not special: an effect of expertise. J Exp Psychol Gen 115(2):107–117

    Article  Google Scholar 

  11. Felzenszwalb P, Girshick R, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645. doi:10.1109/TPAMI.2009.167

    Article  Google Scholar 

  12. Fisher R A (1936) The use of multiple measurements in taxonomic problems. Ann Eugenics 7(2):179–188. doi:10.1111/j.1469-1809.1936.tb02137.x

    Article  Google Scholar 

  13. Hinton G E, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R R (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580

  14. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst:1–9

  15. Liu J, Kanazawa A, Jacobs D, Belhumeur P (2012) Dog breed classification using part localization. In: European conference on computer vision. Springer, pp 172–185

  16. Lowe D (1999) Object recognition from local scale-invariant features. In: International conference on computer vision. doi:10.1109/ICCV.1999.790410, vol 2, pp 1150–1157

  17. Parkhi OM, Vedaldi A, Zisserman A, Jawahar C (2012) Cats and dogs. In: IEEE conference on computer vision and pattern recognition, pp 3498–3505

  18. Pinto N, Stone Z, Zickler T, Cox D (2011) Scaling up biologically-inspired computer vision: a case study in unconstrained face recognition on facebook. In: IEEE computer society conference on computer vision and pattern recognition workshops. doi:10.1109/CVPRW.2011.5981788, pp 35–42

  19. Scapinello K, Yarmey A (1970) The role of familiarity and orientation in immediate and delayed recognition of pictorial stimuli. Psychon Sci 21(6):329–330. doi:10.3758/BF03335807

    Article  Google Scholar 

  20. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y (2014) Overfeat: integrated recognition, localization and detection using convolutional networks. In: International conference on learning representations, CBLS. http://openreview.net/document/d332e77d-459a-4af8-b3ed-55ba

  21. Sirovich L, Kirby M (1987) Low-dimensional procedure for the characterization of human faces. J Opt Soc Am A 4(3):519–524. doi:10.1364/JOSAA.4.000519. http://josaa.osa.org/abstract.cfm?URI=josaa-4-3-519

    Article  Google Scholar 

  22. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition

  23. Turk M, Pentland A (1991) Face recognition using eigenfaces. In: IEEE conference on computer vision and pattern recognition. doi:10.1109/CVPR.1991.139758, pp 586–591

  24. Wang X, Ly V, Sorensen S, Kambhamettu C (2014) Dog breed classification via landmarks. In: IEEE international conference on image processing, IEEE, pp 5237–5241

  25. Wiskott L, Fellous J M, Krüger N, Von Malsburg C D (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19:775–779. doi:10.1109/34.598235

    Article  Google Scholar 

  26. Xu Y, Zhang D, Yang J, Yang J Y (2011) A two-phase test sample sparse representation method for use with face recognition. Circuits and Systems for Video Technology 21(9):1255–1262. doi:10.1109/TCSVT.2011.2138790

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgments

Special thanks to the veterinary doctor Marjorie de Oliveira Franco, along with the Zoonoses Control Center of São José dos Campos - SP, for producing the Snoopybook dataset, and providing it for research. And thanks to Giovani Chiachia for its help on using simple-hp, tips on how to perform the experiments and helping organizing the dataset.

Author information

Authors and Affiliations

  1. LIV Laboratory, Institute of Computing (IC), University of Campinas (Unicamp), Campinas, Brazil

    Thierry Pinheiro Moreira

  2. RECOD Laboratory, Institute of Computing (IC), University of Campinas (Unicamp), Campinas, Brazil

    Mauricio Lisboa Perez, Rafael de Oliveira Werneck & Eduardo Valle

  3. Department of Computer Engineering and Industrial Automation (DCA), School of Electrical and Computer Engineering (FEEC), University of Campinas (Unicamp), Av. Albert Einstein, 400, Campinas, 13083-852, SP, Brazil

    Eduardo Valle

Authors
  1. Thierry Pinheiro Moreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mauricio Lisboa Perez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafael de Oliveira Werneck
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eduardo Valle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thierry Pinheiro Moreira.

Additional information

This work was supported in part by FAPESP, CAPES, CNPq, and SAMSUNG.

T. Moreira, M. Perez, and R. Werneck contributed equally to this work.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Moreira, T.P., Perez, M.L., Werneck, R. et al. Where is my puppy? Retrieving lost dogs by facial features. Multimed Tools Appl 76, 15325–15340 (2017). https://doi.org/10.1007/s11042-016-3824-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3824-1

Keywords

  • Face recognition
  • Dog recognition
  • Deep learning
  • Convolutional networks