Neural Computing and Applications

, Volume 29, Issue 4, pp 1087–1102 | Cite as

Image feature-based affective retrieval employing improved parameter and structure identification of adaptive neuro-fuzzy inference system

  • Dan Wang
  • Ting He
  • Zairan Li
  • Luying Cao
  • Nilanjan Dey
  • Amira S. Ashour
  • Valentina E. Balas
  • Pamela McCauley
  • Yezhi Lin
  • Jiang Xu
  • Fuqian ShiEmail author
Original Article


Affective computing has various challenges especially for features extraction. Semantic information and vocal messages contain much emotional information, while extracting affective from features of images, and affective computing for image dataset are regarded as a promised research direction. This paper developed an improved adaptive neuro-fuzzy inference system (ANFIS) for images’ features extraction. Affective value of valence, arousal, and dominance were the proposed system outputs, where the color, morphology, and texture were the inputs. The least-square and k-mean clustering methods were employed as learning algorithms of the system. This improved model for structure and parameter identification of ANFIS were trained and validated. The training errors of the system for the affective values were tested and compared. Data sources grouping and the ANFIS generating processes were included. In the network training process, the number of input variables and fuzzy subset membership function types has been relative to network model under different affective inputs. Finally, well-established training model was used for testing using International Affective Picture System. The resulting network predicted those affective values, which compared to the expected outputs. The results demonstrated the effect of larger deviation of the individual data. In addition, the relationships between training errors, fuzzy sample set, training data set, function type, and the number of membership functions were illustrated. The proposed model showed the effectiveness for image affective extraction modeling with maximum training errors of 14 %.


Adaptive neuro-fuzzy inference system Structure identification Parameter identification International Affective Picture System k-Mean clustering 



This work was sponsored by Zhejiang Provincial Natural Science Fund under Grant No. (LQ15A010009, Y17F030054) and National Natural Science Foundation of China under Grant No. (51205059).

Compliance with ethical standards

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.


  1. 1.
    Liu N, Dellandréa E, Chen L et al (2013) Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme. Comput Vis Image Underst 117(5):493–512CrossRefGoogle Scholar
  2. 2.
    Nanni L, Brahnam S, Lumini A (2012) Random interest regions for object recognition based on texture descriptors and bag of features. Expert Syst Appl 39(1):973–977CrossRefGoogle Scholar
  3. 3.
    Lo EHS, Pickering MR, Frater MR, Arnold JF (2011) Image segmentation from scale and rotation invariant texture features from the double dyadic dual-tree complex wavelet transform. Image Vis Comput 29(1):15–28CrossRefGoogle Scholar
  4. 4.
    Janssen JH, Ijsselsteijn WA, Westerink JHDM, Tacken P, de Vries G-J (2013) The tell-tale heart: perceived emotional intensity of heartbeats. Int J Synth Emot 4(1):65–91CrossRefGoogle Scholar
  5. 5.
    Ying Yu, Yong C (2009) Based on curvature direction characteristic of image emotional semantic classification. Comput Syst 18(2):121–124Google Scholar
  6. 6.
    Hayashi T, Hagiwara M (1997) An image retrieval system to estimate impression words form images using a neural network. In: Proceedings of the IEEE international conference on systems, man and cybernetics-computational cybernetics and simulation, New York, pp 150–155Google Scholar
  7. 7.
    Yoshida K, Kato T, Yanaru T (1998) Image retrieval system using impression words system. In: Proceedings of the IEEE international conference on systems, man and cybernetics, San Diego, pp 2780–2784Google Scholar
  8. 8.
    Lin HC, Chiu CY, Yang SN (2002) Texture analysis and description in linguistic terms. In: Proceedings of the Asian conference on computer vision, Melbourne, Australia, pp 23–25Google Scholar
  9. 9.
    Dai Y (2004) Intention-based image retrieval with or without a query image. In: Proceedings of the international conference on multimedia modeling, Australia, pp 26–32Google Scholar
  10. 10.
    Abegaz T, Dillon E, Gilbert JE (2015) Exploring affective reaction during user interaction with colors and shapes. Proced Manuf 3:5253–5260CrossRefGoogle Scholar
  11. 11.
    Zhang P, Gong M, Su L, Liu J, Li Z (2016) Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images. ISPRS J Photogram Rem Sens 116:24–41CrossRefGoogle Scholar
  12. 12.
    Yun S, Bin H, Lixin X et al (2015) EEG data for knowledge modeling and emotion recognition. Chin Sci Bull 11(10):1002–1009Google Scholar
  13. 13.
    Lahane P, Sangaiah AK (2015) An approach to EEG based emotion recognition and classification using kernel density estimation. Proced Comput Sci 48:574–581CrossRefGoogle Scholar
  14. 14.
    Castillo O, Melin P, Ramírez E, Soria J (2012) Hybrid intelligent system for cardiac arrhythmia classification with Fuzzy K-Nearest Neighbors and neural networks combined with a fuzzy system. Expert Syst Appl 39(3):2947–2955CrossRefGoogle Scholar
  15. 15.
    Zhang YQ, Kandal A (1998) Compensatory neuro-fuzzy systems with fast learn algorithms. IEEE Trans Neural Netw 9(1):83–105CrossRefGoogle Scholar
  16. 16.
    Adli J, Karim A (2000) Fuzzy-wavelet RBFNN model for freeway incident detection. J Transp Eng 126(6):464–471CrossRefGoogle Scholar
  17. 17.
    Zhijian Z, Zongyuan M (1999) Fuzzy neural network structure and parameters optimization based on genetic algorithm. Hunan Univ Technol (Natural Science) 27(1):26–32Google Scholar
  18. 18.
    Qin Z, Jin C, Li D (2015) ANFIS system applications in the medical field. Chin J Clin Thorac Cardiovasc Hous 22(3):252–256Google Scholar
  19. 19.
    Latifoğlu F (2013) A novel approach to speckle noise filtering based on Artificial Bee Colony algorithm: an ultrasound image application. Comput Methods Programs Biomed 111(3):561–569CrossRefGoogle Scholar
  20. 20.
    Abdulshahed AM, Longstaff AP, Fletcher S, Myers A (2015) Thermal error modelling of machine tools based on ANFIS with fuzzy c-means clustering using a thermal imaging camera. Appl Math Model 39:1837–1852CrossRefGoogle Scholar
  21. 21.
    Jiang H, Kwong C, Siu K, Liu Y (2015) Rough set and PSO-based ANFIS approaches to modeling customer satisfaction for affective product design. Adv Eng Inform 29:727–738CrossRefGoogle Scholar
  22. 22.
    Turkmen I (2011) Efficient impulse noise detection method with ANFIS for accurate image restoration. Int J Electron Commun 65:132–139CrossRefGoogle Scholar
  23. 23.
    Abbaspour S, Fallah A, Lindn M, Gholamhosseini H (2016) A novel approach for removing ECG interferences from surface EMG signals using a combined ANFIS and wavelet. J Electromyogr Kinesiol 26:52–59CrossRefGoogle Scholar
  24. 24.
    Gupta R, ur Rehman Laghari K, Falk TH (2016) Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization. Neurocomputing 174(Part B):875–884CrossRefGoogle Scholar
  25. 25.
    Sharma R, Patterh M (2015) A new pose invariant face recognition system using PCA and ANFIS. Opt Int J Light Electron Opt 126:3483–3487CrossRefGoogle Scholar
  26. 26.
    Bhandari KA, Ramchandra RM (2016) An innovative remote sensing image retrieval techniques based on haar wavelet-LTRP and ANFIS. Proced Comput Sci 79:391–401CrossRefGoogle Scholar
  27. 27.
    Dai W, Han D, Dai Y, Xu D (2015) Emotion recognition and affective computing on vocal social media. Inf Manag 52:777–788CrossRefGoogle Scholar
  28. 28.
    Kaminskas M, Ricci F (2012) Contextual music information retrieval and recommendation: state of the art and challenges. Comput Sci Rev 6:89–119CrossRefGoogle Scholar
  29. 29.
    Bozhkov L, Georgieva P, Santos I, Pereira A, Silva C (2015) EEG-based subject independent affective computing models. Proced Comput Sci 53:375–382CrossRefGoogle Scholar
  30. 30.
    Ward R, Marsden P (2004) Affective computing: problems, reactions and intentions. Interact Comput 16:707–713CrossRefGoogle Scholar
  31. 31.
    Vallverd J, Talanov M, Distefano S, Mazzara M, Tchitchigin A, Nurgaliev I (2016) A cognitive architecture for the implementation of emotions in computing systems. Biol Inspired Cogn Archit 15:34–40Google Scholar
  32. 32.
    Hussain A, Cambria E, Schuller B, Howard N (2014) Affective neural networks and cognitive learning systems for big data analysis. Neural Netw 58:1–3CrossRefGoogle Scholar
  33. 33.
    Rao Y, Li Q, Wenyin L, Wu Q, Quan X (2014) Affective topic model for social emotion detection. Neural Netw 58:29–37CrossRefGoogle Scholar
  34. 34.
    Jiang Y, Liu C, Ma J (2016) BYY harmony learning of t-mixtures with the application to image segmentation based on contourlet texture features. Neurocomputing 188:262–274CrossRefGoogle Scholar
  35. 35.
    Sarafis I, Diou C, Delopoulos A (2016) Online training of concept detectors for image retrieval using streaming clickthrough data. Eng Appl Artif Intell 51:150–162CrossRefGoogle Scholar
  36. 36.
    Haralick RM (1979) Statistical and structural approaches to texture. Proc IEEE 67(5):786–804CrossRefGoogle Scholar
  37. 37.
    Vlachostergiou A, Caridakis G, Kollias S (2014) Investigating context awareness of affective computing systems: a critical approach. Proced Comput Sci 39:91–98CrossRefGoogle Scholar
  38. 38.
    ElAdel A, Zaied M, Amar CB (2016) Fast beta wavelet network-based feature extraction for image copy detection. Neurocomputing 173(Part 2):306–316CrossRefGoogle Scholar
  39. 39.
    Ponti M, Nazar TS, Thum GS (2016) Image quantization as a dimensionality reduction procedure in color and texture feature extraction. Neurocomputing 173(Part 2):385–396CrossRefGoogle Scholar
  40. 40.
    Shahraiyni HT, Sodoudi S, Kerschbaumer A, Cubascha U (2015) A new structure identification scheme for ANFIS and its application for the simulation of virtual air pollution monitoring stations in urban areas. Eng Appl Artif Intell 41:175–182CrossRefGoogle Scholar
  41. 41.
    Chiu SL (1994) Fuzzy model identification based on cluster estimation. J Intell Fuzzy Syst 2(3):267–278Google Scholar
  42. 42.
    Mebrabian A (1995) Framework for a comprehensive description and measurement of emotional states. Genet Soc Gen Psychol Monogr 121(3):3392–3611Google Scholar
  43. 43.
    Mebrabian A (1996) Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr Psychol Dev Learn Personal Soc 14(4):2612–2921Google Scholar
  44. 44.
    Mebrabian A, Wihardja C, Ljunggren E (1997) Emotional correlates of preferences for situation activity combinations in everyday life. Genetic Soc Gen Psychol Monogr 123(4):4612–4771Google Scholar
  45. 45.
    He T, Cao L, Balas VE, McCauley P, Shi F (2016) Curvature manipulation of the spectrum of Valence-Arousal-related fMRI dataset using Gaussian-shaped Fast Fourier Transform and its application to fuzzy KANSEI adjectives modeling. Neurocomputing 174(Part B):1049–1059CrossRefGoogle Scholar
  46. 46.
    Broekens J (2010) Modeling the experience of emotion. Int J Synth Emot 1(1):1–17CrossRefGoogle Scholar
  47. 47.
    Gunes H (2010) Automatic, dimensional and continuous emotion recognition. Int J Synth Emot 1:1Google Scholar
  48. 48.
    Cowie R, McKeown G, Douglas-Cowie E (2012) Tracing emotion: an overview. Int J Synth Emot 3(1):1–17CrossRefGoogle Scholar
  49. 49.
    Mortillaro M, Meuleman B, Scherer KR (2012) Advocating a componential appraisal model to guide emotion recognition. Int J Synth Emot 3(1):18–32CrossRefGoogle Scholar
  50. 50.
    Moral R, Navarro J, Lahoz-Beltra R, Marijuán PC (2014) Cognitive and emotional contents of laughter: framing a new neurocomputational approach. Int J Synth Emot 5(2):31–54CrossRefGoogle Scholar
  51. 51.
    Mikels JA, Fredrickson BL, Larkin GR, Lindberg CM, Maglio SJ, Reuter-Lorenz PA (2005) Emotional category data on images from the International Affective Picture System. Behav Res Methods 37(4):626–630CrossRefGoogle Scholar
  52. 52.
    Machajdik J, Hanbury A (2010) Affective image classification using features inspired by psychology and art theory. In: Proceedings of the 18th ACM international conference on multimedia. ACM, pp. 83–92Google Scholar
  53. 53.
    Bach J (2012) A framework for emergent emotions, based on motivation and cognitive modulators. Int J Synth Emot 3(1):43–63CrossRefGoogle Scholar
  54. 54.
    Grunberg DK, Batula AM, Schmidt EM, Kim YE (2012) Synthetic emotions for humanoids: perceptual effects of size and number of robot platforms. Int J Synth Emot 3(2):68–83CrossRefGoogle Scholar
  55. 55.
    Setiawan NA (2014) Fuzzy decision support system for coronary artery disease diagnosis based on Rough set theory. Int J Rough Sets Data Anal 1(1):65–80MathSciNetCrossRefGoogle Scholar
  56. 56.
    Kim H, Lee I (2015) Combining image databases for affective image classification. In: ACHI 2015: the eighth international conference on advances in computer-human interactions, pp 211–212Google Scholar

Copyright information

© The Natural Computing Applications Forum 2016

Authors and Affiliations

  1. 1.Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and AutomationTianjin UniversityTianjinPeople’s Republic of China
  2. 2.Wenzhou Vocational and Technical CollegeWenzhouPeople’s Republic of China
  3. 3.College of Information and EngineeringWenzhou Medical UniversityWenzhouPeople’s Republic of China
  4. 4.Department of ITTechno India College of TechnologyKolkataIndia
  5. 5.Department of Electronics and Electrical Communications Engineering, Faculty of EngineeringTanta UniversityTantaEgypt
  6. 6.Department of Automation and Applied InformaticsAurel Vlaicu University of AradAradRomania
  7. 7.Department of Industrial Engineering and Management SystemsUniversity of Central FloridaOrlandoUSA
  8. 8.College of Design and InnovationTongji UniversityShanghaiPeople’s Republic of China

Personalised recommendations