Identification and Retrieval of Moth Images Based on Wing Patterns

  • Linan FengEmail author
  • Bir Bhanu
  • John Heraty
Part of the Computational Biology book series (COBO, volume 22)


Moths are important life forms on the planet with approximately 160,000 species discovered. Entomologists in the past need to manually collect moth samples, take digital photos, identify the species, and archive into different categories. This process is time-consuming and requires a lot of human labors. As modern technologies in computer vision and machine learning advance, new algorithms have been developed in recognizing objects in digital images based on their visual attributes. The methods can also be applied to the entomology domain for recognizing biological identities. The Lepidoptera (moths and butterflies) in general can be identified and classified by their body morphological features; wing visual patterns that can be obtained using various image processing approaches in automated diagnostic systems. In this chapter, we describe a system for automated moth species identification and retrieval. The core of the system is a probabilistic model that infers semantically related visual (SRV) attributes from low-level visual features of the moth images in the training set, where moth wings are segmented into information-rich patches from which the local features are extracted, and the SRV attributes are provided by human experts as ground-truth. For the testing images in the database, an automated identification process is evoked to translate the detected salient regions of low-level visual features on the moth wings into meaningful semantic SRV attributes. We further propose a novel network analysis-based approach to explore and utilize the co-occurrence patterns of SRV attributes as contextual cues to improve individual attribute detection accuracy. The effectiveness of the proposed approach is evaluated in automated moth identification and attribute-based image retrieval. In addition, a novel image descriptor called SRV attribute signature is introduced to record the visual and semantic properties of an image and is used to compare image similarity. Experiments are performed on an existing entomology database to illustrate the capabilities of our proposed system.


Image Retrieval Relevance Feedback Scale Invariant Feature Transform Mean Average Precision Salient Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported in part by the National Science Foundation grants 0641076 and 0905671.


  1. 1.
    Carter D (1992) Butterflies and moths. Eyewitness handbooksGoogle Scholar
  2. 2.
    Kerr P, Fisher E, Buffington M (2008) Dome lighting for insect imaging under a microscope. Am Entomol 54:198–200CrossRefGoogle Scholar
  3. 3.
    Buffington M, Gates M (2008) Advanced imaging techniques ii: using a compound microscope for photographing point-mount specimens. Am Entomol 54:222–224CrossRefGoogle Scholar
  4. 4.
    Buffington M, Burks R, McNeil L (2005) Advanced techniques for imaging parasitic Hymenoptera (Insecta). Am Entomol 51:50–56CrossRefGoogle Scholar
  5. 5.
    Wen C, Guyer DE, Li W (2009) Local feature-based identification and classification for orchard insects. Biosyst Eng 104(3):299–307CrossRefGoogle Scholar
  6. 6.
    Francoy TM, Wittmann D, Drauschke M, Müller S, Steinhage V, Bezerra-Laure MAF, Jong DD, Goncalves LS (2008) Identification of africanized honey bees through wing morphometrics: two fast and efficient procedure. Apidologie 39(5):488–494CrossRefGoogle Scholar
  7. 7.
    Yue J, Li Z, Liu L, Fu Z (2011) Content-based image retrieval using color and texture fused features. Math Comput Model 54:1121–1127Google Scholar
  8. 8.
    Bunte K, Biehl M, Jonkman M, Petkov N (2011) Learning effective color features for content based image retrieval in dermatology. Pattern Recogn 44:1892–1902CrossRefGoogle Scholar
  9. 9.
    Singhai N, Shandilya S (2010) A survey on: content based image retrieval systems. Int J Comput Appl 4:22–26Google Scholar
  10. 10.
    Bhanu B, Li R, Heraty J, Murray E (2008) Automated classification of skippers based on parts representation. Am Entomol 228–231Google Scholar
  11. 11.
    Wang J, Lin C, Ji L, Liang A (2012) A new automatic identification system of insect images at the order level. Knowl-Based Syst 33:102–110CrossRefGoogle Scholar
  12. 12.
    Divvala SK, Hoiem D, Hays JH, Efros AA, Hebert M (2009) An empirical study of context in object detection. In: IEEE conference on computer vision and pattern recognition, pp 1271–1278Google Scholar
  13. 13.
    Hanjalic A, Lienhart R, Ma WY, Smith JR (2008) The holy grail of multimedia information retrieval: so close or yet so far away? Proc IEEE 96(4):541–547CrossRefGoogle Scholar
  14. 14.
    Pereira HM, Ferrier S, Walters M, Geller GN, Jongman RHG, Scholes RJ, Bruford MW, Brummitt N, Butchart SHM, Cardoso AC et al (2013) Essential biodiversity variables. Science 339(1):277–278CrossRefGoogle Scholar
  15. 15.
    Bacon SJ, Bacher S, Aebi A (2012) Gaps in border controls are related to quarantine alien insect invasions in Europe. PLoS One 7(10). doi: 10.1371/journal.pone.0047689
  16. 16.
    Kumschick S, Bacher S, Dawson W, Heikkilä J (2012) A conceptual framework for prioritization of invasive alien species for management according to their impact. NeoBiota 15(10):69–100CrossRefGoogle Scholar
  17. 17.
    Steele PR, Pires JC (2011) Biodiversity assessment: State-of-the-art techniques in phylogenomics and species identification. Am J Bot 98(3):415–425CrossRefGoogle Scholar
  18. 18.
    Qing Y, Liu QJ, Yang BJ, Chen HM, Tang J (2012) An insect imaging system to automatic rice light-trap pest identification. J Integr Agr 11:978–985CrossRefGoogle Scholar
  19. 19.
    Ganchev T, Potamitis I, Fakotakis N (2007) Acoustic monitoring of singing insects. In: IEEE international conference on acoustics, speech and signal processing, vol 4Google Scholar
  20. 20.
    Meulemeester TD, Gerbaux P, Boulvin M, Coppée A, Rasmont P (2011) A simplified protocol for bumble bee species identification by cephalic secretion analysis. Int J Study Soc Arthropods 58(5):227–236Google Scholar
  21. 21.
    Joly A, Goëau H, Glotin H, Spampinato C, Bonnet P, Vellinga W, Planque R, Rauber A, Fisher R, Müller H (2014) Lifeclef 2014: multimedia life species identification challenges. Proc LifeCLEF 2014:229–249Google Scholar
  22. 22.
    Wang J, Ji L, Liang A, Yuan D (2011) The identification of butterfly families using content-based image retrieval. Biosyst Eng 111:24–32CrossRefGoogle Scholar
  23. 23.
    Janzen DH, Hallwachs W (2009) Dynamic database for an inventory of the macrocaterpillar fauna, and its food plants and parasitoids, of area de conservacion guanacaste (acg), northwestern costa rica (nn-srnp-nnnnn voucher codes).
  24. 24.
    Sun Y, Bhanu B (2012) Reflection symmetry-integrated image segmentation. IEEE Trans Pattern Anal Mach Intell 34(9):1827–1841CrossRefGoogle Scholar
  25. 25.
    Duan K, Parikh D, Crandall D (2012) Discovering localized attributes for fine-grained recognition. In: IEEE conference on computer vision and pattern recognition, pp 3474–3481Google Scholar
  26. 26.
    Parikh D, Grauman K (2011) Interactively building a discriminative vocabulary of nameable attributes. In: IEEE conference on computer vision and pattern recognition, pp 1681–1688Google Scholar
  27. 27.
    Russell BC, Torralba A, Murphy KP, Freeman WT (2008) Labelme: a database and web-based tool for image annotation. Int J Comput Vis 77:157–173CrossRefGoogle Scholar
  28. 28.
    Prasad VSN, Yegnanarayana B (2004) Finding axes of symmetry from potential fields. IEEE Trans Image Process 13(12):1559–1566CrossRefMathSciNetGoogle Scholar
  29. 29.
    Haralick RM (1979) Statistical and structural approaches to texture. Proc IEEE 67:786–804CrossRefGoogle Scholar
  30. 30.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  31. 31.
    Fu G, Shih F, Wang H (2011) A kernel-based parametric method for conditional density estimation. Pattern Recogn 44(2):284–294CrossRefzbMATHGoogle Scholar
  32. 32.
    Rubner Y, Tomasi C, Guibas LJ (2000) The earth mover’s distance as a metric for image retrieval. Int J Comput Vis 40(2):99–121CrossRefzbMATHGoogle Scholar
  33. 33.
    Yin PY, Bhanu B, Chang KC (2008) Long-term cross-session relevance feedback using virtual features. IEEE Trans Knowl Data Eng 20(3):352–368CrossRefGoogle Scholar
  34. 34.
    Dong A, Bhanu B (2005) Active concept learning in image databases. IEEE Trans Syst Man Cyber Part B 35:450–456Google Scholar
  35. 35.
    Sivic J, Russell B, Efros A, Zisserman A, Freeman W (2005) Discovering object categories in image collections. In: International conference on computer vision, pp 1543–1550Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Center for Research in Intelligent Systems, Bourns College of EngineeringUniversity of California at RiversideRiversideUSA
  2. 2.Entomology DepartmentUniversity of California at RiversideRiversideUSA

Personalised recommendations