Skip to main content
SpringerLink
Log in
Book cover

Emerging Topics in Hardware Security pp 493–525Cite as

Computer Vision for Hardware Security

  • Chapter
  • First Online:

  • 1029 Accesses

Abstract

The application of computer vision to hardware security has the potential to address the limitations of both electrical testing and traditional physical inspection approaches to hardware security. This chapter begins by providing an overview of basic computer vision concepts as well as a description of pipeline tasks. Examples of three hardware security applications where the application of computer vision has been explored, namely integrated circuit counterfeit detection, hardware Trojan detection, and printed circuit board assurance, are discussed in detail. Lastly, the chapter concludes with a discussion of the challenges of applying computer vision to hardware security and provides suggestions for future research directions.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The interested reader can find a more thorough treatment of this topic in [47] and [48].

References

  1. 360 Market Updates: Global PCB PCBA Market. https://www.360marketupdates.com/global-pcb-pcba-market-14845751 (2020). Accessed 8 June 2020

  2. S. Adee, The hunt for the kill switch. IEEE Spectrum 45(5), 34–39 (2008)

    Article  Google Scholar 

  3. M. Alam, H. Shen, N. Asadizanjani, M. Tehranipoor, D. Forte, Impact of X-ray tomography on the reliability of integrated circuits. IEEE Trans. Device Mater. Reliab. 17(1), 59–68 (2017)

    Article  Google Scholar 

  4. N. Asadizanjani, M. Tehranipoor, D. Forte, Counterfeit electronics detection using image processing and machine learning. J. Phys. Conf. Ser. 787, 012023 (2017)

    Article  Google Scholar 

  5. S.I. Association, Others, Winning the battle against counterfeit semiconductor products. Tech. rep. (2013)

    Google Scholar 

  6. M. Azhagan, D. Mehta, H. Lu, S. Agrawal, M. Tehranipoor, D.L. Woodard, N. Asadizanjani, P. Chawla, A review on automatic bill of material generation and visual inspection on PCBs, in ISTFA 2019: Proceedings of the 45th International Symposium for Testing and Failure Analysis (2019), p. 256. books.google.com

    Google Scholar 

  7. C. Bao, D. Forte, A. Srivastava, On application of one-class SVM to reverse engineering-based hardware Trojan detection, in Fifteenth International Symposium on Quality Electronic Design (2014), pp. 47–54

    Google Scholar 

  8. C. Bao, D. Forte, A. Srivastava, On reverse engineering-based hardware Trojan detection. IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. 35(1), 49–57 (2015)

    Article  Google Scholar 

  9. C. Benedek, Detection of soldering defects in Printed Circuit Boards with Hierarchical Marked Point Processes. Pattern Recognit. Lett. 32(13), 1535–1543 (2011). https://doi.org/10.1016/j.patrec.2011.06.006

  10. U.J. Botero, R. Wilson, H. Lu, M.T. Rahman, M.A. Mallaiyan, F. Ganji, N. Asadizanjani, M.M. Tehranipoor, D.L. Woodard, D. Forte, Hardware Trust and Assurance through Reverse Engineering: A Survey and Outlook from Image Analysis and Machine Learning Perspectives (2020). arXiv [eess.IV]

    Google Scholar 

  11. G. Caswell, Counterfeit detection strategies: when to do it/how to do it, in International Symposium on Microelectronics, vol. 2010 (2010), pp. 000227–000233

    Google Scholar 

  12. J. Cassell, Reports of Counterfeit Parts Quadruple Since 2009, Challenging US Defense Industry and National Security. https://technology.informa.com/389481/reports-of-counterfeit-parts-quadruple-since-2009-challenging-us-defense-industry-and-national-security. Accessed 21 July 2020

  13. H.J. Cho, T.H. Park, Wavelet transform based image template matching for automatic component inspection. Int. J. Adv. Manuf. Technol. 50(9–12), 1033–1039 (2010)

    Article  Google Scholar 

  14. F. Courbon, P. Loubet-Moundi, J.J.A. Fournier, A. Tria, A high efficiency hardware trojan detection technique based on fast SEM imaging, in 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE) (2015), pp. 788–793

    Google Scholar 

  15. F. Courbon, P. Loubet-Moundi, J.J.A. Fournier, A. Tria, SEMBA: a SEM based acquisition technique for fast invasive Hardware Trojan detection, in 2015 European Conference on Circuit Theory and Design (ECCTD) (2015), pp. 1–4

    Google Scholar 

  16. M. Cozzi, J.M. Galliere, P. Maurine, Thermal scans for detecting hardware Trojans, in International Workshop on Constructive Side-Channel Analysis and Secure Design (2018), pp. 117–132

    Google Scholar 

  17. M. Crawford, Defense Industrial Base Assessment: Counterfeit Electronics Prepared by U.S. Department of Commerce Bureau of Industry and Security Office of Technology Evaluation (2010). http://bis.doc.gov/defenseindustrialbaseprograms/

  18. A.J. Crispin, V. Rankov, Automated inspection of PCB components using a genetic algorithm template-matching approach. Int. J. Adv. Manuf. Technol. 35(3), 293–300 (2007). https://doi.org/10.1007/s00170-006-0730-0

    Article  Google Scholar 

  19. A.M. Darwish, A.K. Jain, A rule based approach for visual pattern inspection. IEEE Trans. Pattern Anal. Mach. Intell. 10(1), 56–68 (1988). https://doi.org/10.1109/34.3867

    Article  Google Scholar 

  20. N. Dave, V. Tambade, B. Pandhare, S. Saurav, PCB defect detection using image processing and embedded system. Int. Res. J. Eng. Technol. 3(5), 1897–1901 (2016)

    Google Scholar 

  21. A.R. de Mello, M.R. Stemmer, Inspecting surface mounted devices using k nearest neighbor and Multilayer Perceptron, in 2015 IEEE 24th International Symposium on Industrial Electronics (ISIE) (2015), pp. 950–955. https://doi.org/10.1109/ISIE.2015.7281599

  22. J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, L. Fei-Fei, ImageNet: a large-scale hierarchical image database, in CVPR09 (2009)

    Google Scholar 

  23. N. Dhanachandra, K. Manglem, Y.J. Chanu, Image segmentation using K-means clustering algorithm and subtractive clustering algorithm. Proc. Comput. Sci. 54, 764–771 (2015). https://doi.org/10.1016/j.procs.2015.06.090

    Article  Google Scholar 

  24. C. Fonseka, J. Jayasinghe, Feature extraction and template matching algorithm classification for PCB fiducial verification. J. Achiev. Mater. Manuf. Eng. 86(1), 14–32 (2018)

    Google Scholar 

  25. P. Ghosh, R.S. Chakraborty, Counterfeit IC detection by image texture analysis, in 2017 Euromicro Conference on Digital System Design (DSD), pp. 283–286 (2017)

    Google Scholar 

  26. P. Ghosh, R.S. Chakraborty, Recycled and remarked counterfeit integrated circuit detection by image-processing-based package texture and indent analysis. IEEE Trans. Ind. Inf. 15(4), 1966–1974 (2018)

    Article  Google Scholar 

  27. P. Ghosh, D. Forte, D.L. Woodard, R.S. Chakraborty, Automated detection of pin defects on counterfeit microelectronics, in ISTFA 2018: Proceedings from the 44th International Symposium for Testing and Failure Analysis (2018), p. 57

    Google Scholar 

  28. P. Ghosh, A. Bhattacharya, D. Forte, R.S. Chakraborty, Automated defective pin detection for recycled microelectronics identification. J. Hardw. Syst. Secur. 3(3), 250–260 (2019)

    Article  Google Scholar 

  29. P. Ghosh, U. Botero, F. Ganji, D. Woodard, R.S. Chakraborty, D. Forte, Automated detection and localization of counterfeit chip defects by texture analysis in infrared (IR) domain, in IEEE International Conference on Physical Assurance and Inspection of Electronics (PAINE) (2020)

    Google Scholar 

  30. R.C. Gonzalez, R.E. Woods, Digital Image Processing, 4th edn. (Pearson, London, 2017)

    Google Scholar 

  31. E. Guerra, J.R. Villalobos, A three-dimensional automated visual inspection system for SMT assembly. Comput. Ind. Eng. 40(1–2), 175–190 (2001)

    Article  Google Scholar 

  32. U. Guin, D. DiMase, M. Tehranipoor, A comprehensive framework for counterfeit defect coverage analysis and detection assessment. J. Electron. Test. 30(1), 25–40 (2014)

    Article  Google Scholar 

  33. U. Guin, K. Huang, D. DiMase, J.M. Carulli, M. Tehranipoor, Y. Makris, Counterfeit integrated circuits: a rising threat in the global semiconductor supply chain. Proc. IEEE 102(8), 1207–1228 (2014)

    Article  Google Scholar 

  34. T. Jabid, M.H. Kabir, O. Chae, Local directional pattern (LDP) for face recognition, in 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE) (2010), pp. 329–330

    Google Scholar 

  35. M.J. Jianbo Shi, Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000). https://doi.org/10.1109/34.868688

    Article  Google Scholar 

  36. B.C. Jiang, C.C. Wang, Y.N. Hsu, Machine vision and background remover-based approach for PCB solder joints inspection. Int. J. Prod. Res. 45(2), 451–464 (2007). https://doi.org/10.1080/00207540600607184

    Article  Google Scholar 

  37. R. Johnson, The Navy Bought Fake Chinese Microchips That Could Have Disarmed U.S. Missiles. Business Insider (2011)

    Google Scholar 

  38. K.I. Laws, Rapid texture identification, in Image Processing for Missile Guidance, vol. 0238 (1980), pp. 376– 381

    Google Scholar 

  39. D.T. Lee, A computerized automatic inspection system for complex printed thick film patterns, in Applications of Electronic Imaging Systems, vol. 0143 (International Society for Optics and Photonics, Bellingham, WA, 1978), pp. 172–177. https://doi.org/10.1117/12.956563

  40. W. Li, B. Esders, M. Breier, SMD segmentation for automated PCB recycling, in 2013 11th IEEE International Conference on Industrial Informatics (INDIN). ieeexplore.ieee.org (2013), pp. 65–70. https://doi.org/10.1109/INDIN.2013.6622859

  41. D.U. Lim, Y.G. Kim, T.H. Park, SMD classification for automated optical inspection machine using convolution neural network, in 2019 Third IEEE International Conference on Robotic Computing (IRC) (2019), pp. 395–398

    Google Scholar 

  42. H. Lu, D. Mehta, O.D. Paradis, N. Asadizanjani, M. Tehranipoor, D.L. Woodard, FICS-PCB: a multi-modal image dataset for automated printed circuit board visual inspection (2020)

    Google Scholar 

  43. O.G. Luo Juan, A comparison of SIFT, PCA-SIFT and SURF. Int. J. Image Process 3(4), 143–152 (2009)

    Google Scholar 

  44. K. Mahmood, P.L. Carmona, S. Shahbazmohamadi, F. Pla, B. Javidi, Real-time automated counterfeit integrated circuit detection using X-ray microscopy. Appl. Opt. 54(13), D25–D32 (2015)

    Article  Google Scholar 

  45. P.S. Malge, R.S. Nadaf, PCB defect detection, classification and localization using mathematical morphology and image processing tools. Int. J. Comput. Appl. Technol. 87(9), 40–45 (2014)

    Google Scholar 

  46. D. Mehta, H. Lu, O.D. Paradis, M. Azhagan, T. Rahman, P. Chawla, D.L. Woodard, M. Tehranipoor, N. Asadizanjani, The big hack explained: detection and prevention of PCB supply chain implants. ACM J. Emerg. Technol. Comput. Syst. https://doi.org/10.1145/3401980

  47. M. Moganti, F. Ercal, C.H. Dagli, S. Tsunekawa, Automatic PCB inspection algorithms: a survey. Comput. Vis. Image Underst. 63(2), 287–313 (1996). https://doi.org/10.1006/cviu.1996.0020

    Article  Google Scholar 

  48. T.S. Newman, A.K. Jain, A survey of automated visual inspection. Comput. Vis. Image Underst. 61(2), 231–262 (1995). https://doi.org/10.1006/cviu.1995.1017

    Article  Google Scholar 

  49. A.N. Nowroz, K. Hu, F. Koushanfar, S. Reda, Novel techniques for high-sensitivity hardware Trojan detection using thermal and power maps. IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. 33(12), 1792–1805 (2014)

    Article  Google Scholar 

  50. M. Piccardi, Background subtraction techniques: a review, in 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), vol. 4. ieeexplore.ieee.org (2004), pp. 3099–3104. https://doi.org/10.1109/ICSMC.2004.1400815

  51. B.A. Procurement, Printed Circuit Board (PCB) Forecast Report for 2020. Tech. rep. (2019)

    Google Scholar 

  52. J. Li, J. Gu, Z. Huang, J. Wen, Application research of improved YOLO V3 algorithm in PCB electronic component detection. NATO Adv. Sci. Inst. Ser. E Appl. Sci. 9(18), 3750 (2019) https://doi.org/10.3390/app9183750

  53. M. Reza, Z. Chen, D. Crandall, Deep neural network based detection and verification of microelectronic images. J. Hardw. Syst. Secur. 4, 44–54 (2020)

    Article  Google Scholar 

  54. H. Rob, Counterfeit electronic component detection. https://www.aeri.com/counterfeit-electronic-component-detection/. Accessed 21 July 2020

  55. K. Rosenfeld, R. Karri, Attacks and defenses for JTAG. IEEE Des. Test Comput. 27(1), 36–47 (2010)

    Article  Google Scholar 

  56. S. Shahbazmohamadi, E.H. Jordan, Optimizing an SEM-based 3D surface imaging technique for recording bond coat surface geometry in thermal barrier coatings. Meas. Sci. Technol. 23(12), 125601 (2012)

    Google Scholar 

  57. S. Shahbazmohamadi, D. Forte, M. Tehranipoor, Advanced physical inspection methods for counterfeit detection, in Proceedings of International Symposium for Testing and Failure Analysis (ISFTA) (2014), pp. 55–64

    Google Scholar 

  58. S. Shahbazmohamadi, D. Forte, M. Tehranipoor, Advanced physical inspection methods for counterfeit IC detection, in ISTFA 2014: Conference Proceedings from the 40th International Symposium for Testing and Failure Analysis (2014), p. 55

    Google Scholar 

  59. Q. Shi, N. Vashistha, H. Lu, H. Shen, B. Tehranipoor, D.L. Woodard, N. Asadizanjani, Golden gates: a new hybrid approach for rapid hardware trojan detection using testing and imaging, in 2019 IEEE International Symposium on Hardware Oriented Security and Trust (HOST) (2019), pp. 61–71

    Google Scholar 

  60. Y. Shi, C.W. Ting, B.H. Gwee, Y. Ren, A highly efficient method for extracting FSMs from flattened gate-level netlist, in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (2010), pp. 2610–2613

    Google Scholar 

  61. Y.N. Sun, C.T. Tsai, A new model-based approach for industrial visual inspection. Pattern Recognit. 25(11), 1327–1336 (1992). https://doi.org/10.1016/0031-3203(92)90145-9

  62. C. Szymanski, M.R. Stemmer, Automated PCB inspection in small series production based on SIFT algorithm, in 2015 IEEE 24th International Symposium on Industrial Electronics (ISIE). ieeexplore.ieee.org (2015), pp. 594–599. https://doi.org/10.1109/ISIE.2015.7281535

  63. M. Tehranipoor, F. Koushanfar, A survey of hardware Trojan taxonomy and detection. IEEE Des. Test Comput. 27(1), 10–25 (2010)

    Article  Google Scholar 

  64. N. Vashistha, H. Lu, Q. Shi, M.T. Rahman, H. Shen, D.L. Woodard, N. Asadizanjani, M. Tehranipoor, Trojan scanner: detecting hardware trojans with rapid SEM imaging combined with image processing and machine learning, in ISTFA 2018: Proceedings from the 44th International Symposium for Testing and Failure Analysis (2018), p. 256

    Google Scholar 

  65. N. Vashistha, M.T. Rahman, H. Shen, D.L. Woodard, N. Asadizanjani, M. Tehranipoor, Detecting hardware Trojans inserted by untrusted foundry using physical inspection and advanced image processing. J. Hardw. Syst. Secur. 2(4), 333–344 (2018)

    Article  Google Scholar 

  66. T.J. Wagner, Hierarchical layout verification. IEEE Des. Test Comput. 2(1), 31–37 (1985)

    Article  Google Scholar 

  67. D.Z. Wang, C.H. Wu, A. Ip, C.Y. Chan, D.W. Wang, Fast multi-template matching using a particle swarm optimization algorithm for PCB inspection, in Applications of Evolutionary Computing (Springer, Berlin, Heidelberg, 2008), pp. 365–370. https://doi.org/10.1007/978-3-540-78761-7_39

    Google Scholar 

  68. F. Wu, X. Zhang, Feature-extraction-based inspection algorithm for IC solder joints. IEEE Trans. Compon. Packaging Manuf. Technol. 1(5), 689–694 (2011). https://doi.org/10.1109/TCPMT.2011.2118208

    Article  Google Scholar 

  69. H.H. Wu, X.M. Zhang, S.L. Hong, A visual inspection system for surface mounted components based on color features. in 2009 International Conference on Information and Automation (2009), pp. 571–576

    Google Scholar 

  70. H. Wu, G. Feng, H. Li, X. Zeng, Automated visual inspection of surface mounted chip components, in 2010 IEEE International Conference on Mechatronics and Automation (2010), pp. 1789–1794

    Google Scholar 

  71. L. Yang, X. Li, H. Li, Hardware Trojan detection method based on time feature of chip temperature, in 2020 10th Annual Computing and Communication Workshop and Conference (CCWC) (2020), pp. 1029–1032

    Google Scholar 

  72. C.H. Yeh, T.C. Shen, F.C. Wu, A case study: passive component inspection using a 1D wavelet transform. Int. J. Adv. Manuf. Technol. 22(11), 899–910 (2003). https://doi.org/10.1007/s00170-003-1608-z

    Article  Google Scholar 

  73. S. Youn, Y. Lee, T. Park, Automatic classification of SMD packages using neural network, in 2014 IEEE/SICE International Symposium on System Integration (2014), pp. 790–795

    Google Scholar 

  74. Z. Zeng, Z.M. Li, Z. Zheng, Extracting PCB components based on color distribution of highlight areas. Comput. Sci. Inf. Syst. 7(1), 13–30 (2010)

    Article  Google Scholar 

  75. B. Zhou, A. Aksoylar, K. Vigil, R. Adato, J. Tan, B. Goldberg, M.S. Ünlü, A. Joshi, Hardware Trojan detection using backside optical imaging. IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. (2020). https://doi.org/10.1109/TCAD.2020.2991680

Download references

Author information

Authors and Affiliations

  1. University of Florida, Gainesville, FL, USA

    Hangwei Lu, Daniel E. Capecci, Pallabi Ghosh, Domenic Forte & Damon L. Woodard

Authors
  1. Hangwei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel E. Capecci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pallabi Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Domenic Forte
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Damon L. Woodard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Damon L. Woodard .

Editor information

Editors and Affiliations

  1. University of Florida, Gainesville, FL, USA

    Mark Tehranipoor

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lu, H., Capecci, D.E., Ghosh, P., Forte, D., Woodard, D.L. (2021). Computer Vision for Hardware Security. In: Tehranipoor, M. (eds) Emerging Topics in Hardware Security . Springer, Cham. https://doi.org/10.1007/978-3-030-64448-2_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-64448-2_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-64447-5

  • Online ISBN: 978-3-030-64448-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation