Journal of Real-Time Image Processing

, Volume 15, Issue 2, pp 249–264 | Cite as

On-chip real-time feature extraction using semantic annotations for object recognition

  • Ying-Hao YuEmail author
  • Tsu-Tian Lee
  • Pei-Yin Chen
  • Ngaiming Kwok
Original Research Paper


Describing image features in a concise and perceivable manner is essential to focus on candidate solutions for classification purpose. In addition to image recognition with geometric modeling and frequency domain transformation, this paper presents a novel 2D on-chip feature extraction named semantics-based vague image representation (SVIR) to reduce the semantic gap of content-based image retrieval. The development of SVIR aims at successively deconstructing object silhouette into intelligible features by pixel scans and then evolves and combines piecewise features into another pattern in a linguistic form. In addition to semantic annotations, SVIR is free of complicated calculations so that on-chip designs of SVIR can attain real-time processing performance without making use of a high-speed clock. The effectiveness of SVIR algorithm was demonstrated with timing sequences and real-life operations based on a field-programmable-gate-array (FPGA) development platform. With low hardware resource consumption on a single FPGA chip, the design of SVIR can be used on portable machine vision for ambient intelligence in the future.


Semantics-based vague image representation (SVIR) Bipolar image encoding Vertical evolution Lateral combination 



Supports from Ministry of Science and Technology, funded by the government of Taiwan under Grant NSC 99-2221-E-027-057-MY3, Ministry of Education Taiwan for the Top University Project to the National Cheng Kung University (NCKU), and Mr. Yao-Long Cai are gratefully acknowledged.

Supplementary material

Supplementary material 1 (WMV 1734 kb)


  1. 1.
    Bezdek, J.C., Keller, J., Krisnapuram, R., Pal, N.R.: Fuzzy models and algorithms for pattern recognition and image processing. Springer (2005)Google Scholar
  2. 2.
    Nixon, M.S., Aguado, A.S.: Feature extraction and image processing. Academic Press, London (2008)Google Scholar
  3. 3.
    Treiber, M.: An introduction to object recognition. Springer, New York (2010)CrossRefzbMATHGoogle Scholar
  4. 4.
    Li, S.Z., Jain, A.K.: Handbook of face recognition. Springer, New York (2011)CrossRefzbMATHGoogle Scholar
  5. 5.
    Arth, C., Bischof, H.: Real-time object recognition using local features on a DSP-based embedded system. J. Real Time Image Proc. 3, 233–253 (2008)CrossRefGoogle Scholar
  6. 6.
    Lin, C.T., Yeh, C.M., Liang, S.F., Chung, J.F., Kumar, N.: Support-vector-based fuzzy neural network for pattern classification. IEEE Trans. Fuzzy Syst. 14(1), 31–41 (2006)CrossRefGoogle Scholar
  7. 7.
    Espejo, P.G., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(2), 121–144 (2010)CrossRefGoogle Scholar
  8. 8.
    Berretti, S., Bimbo, A.D., Pala, P.: Retrieval by shape similarity with perceptual distance and effective indexing. IEEE Trans. Multimed. 2(4), 225–239 (2000)CrossRefGoogle Scholar
  9. 9.
    Shotton, J., Black, A., Cipolla, R.: Multi-scale categorical object recognition using contour fragments. IEEE Trans. Pattern Anal. Mach. Intell. 30(7), 1270–1281 (2008)CrossRefGoogle Scholar
  10. 10.
    Chung, Y., Prasanna, V.K.: Parallelizing image feature extraction on coarse-grain machines. IEEE Trans. Pattern Anal. Mach. Intell. 20(12), 1389–1394 (1998)CrossRefGoogle Scholar
  11. 11.
    Abbasi, S., Mokhtarian, F., Kittler, J.: Curvature scale space image in shape similarity retrieval. Multi Systems 7, 467–476 (1999)CrossRefGoogle Scholar
  12. 12.
    Bai, X., Latecki, L.J., Liu, W.Y.: Skeleton pruning by contour partitioning with discrete curve evolution. IEEE Trans. Pattern Anal Mach. Intell. 29(30), 449–462 (2007)CrossRefGoogle Scholar
  13. 13.
    Khan, S., Sanches, J., Ventura, R.: Robust band profile extraction using constrained nonparametric machine-learning technique. IEEE Trans. Biomed. Eng. 57(10), 2587–2591 (2010)CrossRefGoogle Scholar
  14. 14.
    Mahmoodi, S., Sharif, B.S.: Contour evolution scheme for variation image segmentation and smoothing. IET J. Dig. Object Identif. 1(3), 287–294 (2007)Google Scholar
  15. 15.
    Lai, H.C., Savvides, M., Chen, T.: Proposed FPGA hardware architecture for high frame rate (≥100fps) face detection using feature cascade classifiers. In: 1st IEEE International Conference on Biometrics: Theory, Applications, and Systems, 27–29 Sept 2007, pp. 1–6 (2007)Google Scholar
  16. 16.
    Yao, M., Yi, W., Zhu, R., Chen, R.: Semantic image retrieval based on multiple-instance learning. In: 10th IEEE International Conference on Data Mining Workshops (ICDMW), 13–13 Dec 2010, pp. 905–910 (2010)Google Scholar
  17. 17.
    Wang, B., Zhang, X., Zhao, X.Y., Zhang, Z. D., Zhang, H.Z.: A semantic description for content-based image retrieval. In: International Conference on Machine Learning and Cybernetics, 12–15 July 2008, pp. 2466–2469 (2008)Google Scholar
  18. 18.
    Akakin, H.C., Gurcan, M.N.: Content-based microscopic image retrieval system for multi-image queries. IEEE Trans Inf. Technol. Biomed. 16(4), 758–769 (2012)CrossRefGoogle Scholar
  19. 19.
    Liu, Y., Zhang, D., Lu, G., Ma, W.Y.: A survey of content-based image retrieval with high-level semantics. Pattern Recogn. 40, 262–282 (2007)CrossRefzbMATHGoogle Scholar
  20. 20.
    Rahman, M.M., Desai, B.C., Bhattacharya, P.: A feature level fusion in similarity matching to content-based image retrieval. In: 9th International Conference on Information Fusion (Fusion), 10–13 July 2006, pp. 1–6 (2006)Google Scholar
  21. 21.
    Mostefaoui, S.K., Maamar, Z., Giaglis, G.M.: Advanced in ubiquitous computing: future paradigms and directions. IGI Publishing, Hershey (2008)CrossRefGoogle Scholar
  22. 22.
    Remagnino, P., Foresti, G.L.: Ambient intelligence: a new multidisciplinary paradigm. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 35(1), 1–6 (2005)CrossRefGoogle Scholar
  23. 23.
    Kim, T., Choi, Y., Ham, S., Chung, J.Y., Hyun, J., Li, J., Hong, J. W-K.: Monitoring and detecting abnormal behavior in mobile cloud infrastructure. In: IEEE Network Operations and Management Symposium. pp. 1303–1310 (2012)Google Scholar
  24. 24.
    Yu, Y.H., Kwok, N., Ha, Q.P.: Color tracking for multiple robot control using a system-on-programmable-chip. Autom Constr. 20, 669–676 (2011)CrossRefGoogle Scholar
  25. 25.
    Tsai, Y.K., Jian, L.Y., Hsu, P.L., Wang, B.C.: Implementation of autonomous vehicles with the hough transform and fuzzy control. In: SICE Annual Conference, pp. 2095–2101 (2007)Google Scholar
  26. 26.
    Halder, S., Bhattacharjee, D., Nasipuri, M., Basu, D. K.: A fast FPGA based architecture for sobel edge detection. In: Progress in VLSI Design and Test, vol. 7373, pp. 300–306 (2012)Google Scholar
  27. 27.
    Zhong, F., Capson, D.W., Schuurman, D.: Parallel architecture for PCA image feature detection using FPGA. In: International Conference on Electrical and Computer Engineering (CCECE), 4–7 May 2008, pp. 1341–1344 (2008)Google Scholar
  28. 28.
    Bahoura, M., Ezzaidi, H.: FPGA implementation of a feature extraction technique based on fourier transform. In: 24th International Conference on Microelectronics (ICM), 16–20 Dec 2012, pp. 1–4 (2012)Google Scholar
  29. 29.
    Zhou, X., Tomagou, N., Ito, Y., Nakano, K.: Efficient Hough transform on the FPGA using DSP slices and block RAMs. In: 27th International Symposium on Parallel and Distributed Processing Workshops and PhD Forum (IPDPSW), pp. 771–778 (2013)Google Scholar
  30. 30.
    Yao, L., Feng, H., Zhu, Y., Jiang, Z., Zhao, D., Feng, W.: An architecture of optimised SIFT feature detection for an FPGA implementation of an image matcher. In: International Conference on Field-programmable Technology (FPT), 9–11 Dec 2009, pp. 30–37 (2009)Google Scholar
  31. 31.
    Latecki, L.J., Lakämper, R.: Convexity rule for shape decomposition based on discrete contour evolution. Comput. Vis. Image Underst. 73(3), 441–454 (1999)CrossRefGoogle Scholar
  32. 32.
    Chowhan, S.S., Shinde, G.N.: Evaluation of statistical feature encoding techniques on iris images. In: 29th World Congress on Computer Science and Information Engineering, Mar 31–Apr 2 2009, pp. 71–75 (2009)Google Scholar
  33. 33.
    Buciu, I., Gacsadi, A.: Spatiotemporal facial features encoding for facial expression analysis in image sequences. In: 10th International Symposium on Signals, Circuits and Systems (ISSCS), June 30–July 1 2011, pp. 1–4 (2011)Google Scholar
  34. 34.
    Huang, C., Ai, H., Li, Y., Lao, S.: High-performance rotation invariant multiview face detection. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 671–686 (2007)CrossRefGoogle Scholar
  35. 35.
    Mörwald, T., Prankl, J.: Extensions for robust object tracking with a Monte Carlo particle filter. J. Real Time Image Process. p. 15 (online)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Ying-Hao Yu
    • 1
    Email author
  • Tsu-Tian Lee
    • 2
    • 3
  • Pei-Yin Chen
    • 4
  • Ngaiming Kwok
    • 5
  1. 1.Department of Electrical EngineeringNational Chung Cheng UniversityMin-Hsiung TownshipTaiwan
  2. 2.Department of Electrical EngineeringTamkang UniversityNew TaipeiTaiwan
  3. 3.Department of Electrical EngineeringChung Yuan Christian UniversityChung Li CityTaiwan
  4. 4.Department of Computer Science and Information EngineeringNational Cheng Kung UniversityTainanTaiwan
  5. 5.School of Mechanical and Manufacturing EngineeringThe University of New South WalesSydneyAustralia

Personalised recommendations