Engineering the Perception of Recognition Through Interactive Raw Primal Sketch by HNFGS and CNN-MRF

  • Apurba Das
  • Nitin Ajithkumar
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 703)


The impression of a scene on human brain, specifically the primary visual cortex, is still a far-reached goal by the computer vision research community. This work is a proposal of a novel system to engineer the human perception of recognizing a subject of interest. This end-to-end solution implements all the stages from entropy-based unbiased cognitive interview to the final reconstruction of human perception in terms of machine sketch in the framework of forensic sketch of suspects. The lower mid-level vision as designed behaviorally in primary visual cortex honoring the scale-space concept of object identification has been modeled by hierarchical 2D filters, namely hierarchical neuro-visually inspired figure-ground segregation (HNFGS) for interactive sketch rendering. The aforementioned human–machine interaction is twofold: in gross structural design layer and finer/granular modification of the pre-realized digital perception. Pre-realized sketches are formed learning the characteristics of human artists while sketching an object through integrated framework of deep convolutional neural network (D-CNN) and Markov Random field (MRF). After few iterations of interactive fine-tuning of the sketch, a psycho-visual experiment has been designed and performed to evaluate the feasibility and effectiveness of the proposed algorithm.


  1. 1.
    Ullman, S.: High Level Vision MIT Press, Cambridge, Massachussets, 1996.Google Scholar
  2. 2.
    Paterson, A., Squad, C. I., Police, V.: Computerised facial construction and reconstruction. Proceedings of the Asia Pacific Police Technology Conference, 135–144 (1991)Google Scholar
  3. 3.
    Frowd, C. D., Hancock, P. J., Carson, D.: EvoFIT: A holistic, evolutionary facial imaging technique for creating composites. ACM Transactions on applied perception (TAP)1, no. 1, 19–39 (2004)Google Scholar
  4. 4.
    Laughery, K. R., Fowler, R. H.: Sketch artist and Identi-kit procedures for recalling faces. Journal of Applied Psychology 65, no. 3, 307 (1980)Google Scholar
  5. 5.
    Willis, G. B.: Cognitive interviewing and questionnaire design: a training manual. US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics(1994)Google Scholar
  6. 6.
    Zhang, L., Lin, L., Wu, X., Ding, S., Zhang, L. : End-to-end photo-sketch generation via fully convolutional representation learning. 5th ACM on International Conference on Multimedia Retrieval, 627–634 (2015)Google Scholar
  7. 7.
    Whitbeck, M.,Guo, H.: Multiple Landmark Warping Using Thin-plate Splines. IPCV, 6, 256–263 (2006)Google Scholar
  8. 8.
    Das, A.: Digital Communication: Principles and system modelling. Springer Science and Business Media, 169–172 (2010)Google Scholar
  9. 9.
    Parua, S; Das, A; Mazumdar D.; Mitra S.: Determination of Feature Hierarchy from Gabor and SIFT Features for Face Recognition. Second International Conference on Emerging Applications of Information Technology, 2011, pp. 257–260Google Scholar
  10. 10.
    Wise, R. A., Fishman, C. S., Safer, M. A.: How to analyze the accuracy of eyewitness testimony in a criminal case., Conn. L. Rev., 42, 435 (2009)Google Scholar
  11. 11.
    Marr, D.: Vision: A computational investigation into the human representation and processing of visual information. MIT press, 2010.Google Scholar
  12. 12.
    D. Marr and E. Hildreth. Theory of edge detection. In Proceedings of the Royal Society of London, 1980, 207, 187217.Google Scholar
  13. 13.
    R. W. Rodieck and J. Stone. Analysis of receptive fields of cat retinal ganglion cells. Journal of Neurophysiology, 1965, 28:833849.Google Scholar
  14. 14.
    Ikeda, H. and Wright, J. H.: Functional organization of the periphery effect in retinal ganglion cells Vision Research, 1972, 12, 1857–1879Google Scholar
  15. 15.
    Ghosh, K., Roy, A.: Neuro-visually inspired figure-ground segregation. International Conference on Image Information Processing (ICIIP), 1–6 (2011)Google Scholar
  16. 16.
    Lindeberg: Scale-space theory: A basic tool for analyzing structures at different scales. Journal of Applied Statistics, 21(2):224270, 1994.Google Scholar
  17. 17.
    Das, A. and Ghosh, K.: Enhancing face matching in a suitable binary environment. International Conference on Image Information Processing (ICIIP), 1–6 (2011)Google Scholar
  18. 18.
    Das, A., Roy, A., and Ghosh, K.: Proposing a CNN Based Architecture of Mid-Level Vision for Feeding the WHERE and WHAT Pathways in the Brain, FANCCO, LNCS 7076/2011, pp. 559–568.Google Scholar
  19. 19.
    Wu, Z., Lin, D., Tang, X.: Deep Markov Random Field for Image Modeling. 14th European Conference on Computer Vision (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Embedded Innovation Lab.Tata Consultancy ServicesBengaluruIndia
  2. 2.Amrita School of EngineeringAmrita VishwavidyapeethamKollamIndia

Personalised recommendations