Abstract
The application of computer vision to hardware security has the potential to address the limitations of both electrical testing and traditional physical inspection approaches to hardware security. This chapter begins by providing an overview of basic computer vision concepts as well as a description of pipeline tasks. Examples of three hardware security applications where the application of computer vision has been explored, namely integrated circuit counterfeit detection, hardware Trojan detection, and printed circuit board assurance, are discussed in detail. Lastly, the chapter concludes with a discussion of the challenges of applying computer vision to hardware security and provides suggestions for future research directions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
360 Market Updates: Global PCB PCBA Market. https://www.360marketupdates.com/global-pcb-pcba-market-14845751 (2020). Accessed 8 June 2020
S. Adee, The hunt for the kill switch. IEEE Spectrum 45(5), 34–39 (2008)
M. Alam, H. Shen, N. Asadizanjani, M. Tehranipoor, D. Forte, Impact of X-ray tomography on the reliability of integrated circuits. IEEE Trans. Device Mater. Reliab. 17(1), 59–68 (2017)
N. Asadizanjani, M. Tehranipoor, D. Forte, Counterfeit electronics detection using image processing and machine learning. J. Phys. Conf. Ser. 787, 012023 (2017)
S.I. Association, Others, Winning the battle against counterfeit semiconductor products. Tech. rep. (2013)
M. Azhagan, D. Mehta, H. Lu, S. Agrawal, M. Tehranipoor, D.L. Woodard, N. Asadizanjani, P. Chawla, A review on automatic bill of material generation and visual inspection on PCBs, in ISTFA 2019: Proceedings of the 45th International Symposium for Testing and Failure Analysis (2019), p. 256. books.google.com
C. Bao, D. Forte, A. Srivastava, On application of one-class SVM to reverse engineering-based hardware Trojan detection, in Fifteenth International Symposium on Quality Electronic Design (2014), pp. 47–54
C. Bao, D. Forte, A. Srivastava, On reverse engineering-based hardware Trojan detection. IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. 35(1), 49–57 (2015)
C. Benedek, Detection of soldering defects in Printed Circuit Boards with Hierarchical Marked Point Processes. Pattern Recognit. Lett. 32(13), 1535–1543 (2011). https://doi.org/10.1016/j.patrec.2011.06.006
U.J. Botero, R. Wilson, H. Lu, M.T. Rahman, M.A. Mallaiyan, F. Ganji, N. Asadizanjani, M.M. Tehranipoor, D.L. Woodard, D. Forte, Hardware Trust and Assurance through Reverse Engineering: A Survey and Outlook from Image Analysis and Machine Learning Perspectives (2020). arXiv [eess.IV]
G. Caswell, Counterfeit detection strategies: when to do it/how to do it, in International Symposium on Microelectronics, vol. 2010 (2010), pp. 000227–000233
J. Cassell, Reports of Counterfeit Parts Quadruple Since 2009, Challenging US Defense Industry and National Security. https://technology.informa.com/389481/reports-of-counterfeit-parts-quadruple-since-2009-challenging-us-defense-industry-and-national-security. Accessed 21 July 2020
H.J. Cho, T.H. Park, Wavelet transform based image template matching for automatic component inspection. Int. J. Adv. Manuf. Technol. 50(9–12), 1033–1039 (2010)
F. Courbon, P. Loubet-Moundi, J.J.A. Fournier, A. Tria, A high efficiency hardware trojan detection technique based on fast SEM imaging, in 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE) (2015), pp. 788–793
F. Courbon, P. Loubet-Moundi, J.J.A. Fournier, A. Tria, SEMBA: a SEM based acquisition technique for fast invasive Hardware Trojan detection, in 2015 European Conference on Circuit Theory and Design (ECCTD) (2015), pp. 1–4
M. Cozzi, J.M. Galliere, P. Maurine, Thermal scans for detecting hardware Trojans, in International Workshop on Constructive Side-Channel Analysis and Secure Design (2018), pp. 117–132
M. Crawford, Defense Industrial Base Assessment: Counterfeit Electronics Prepared by U.S. Department of Commerce Bureau of Industry and Security Office of Technology Evaluation (2010). http://bis.doc.gov/defenseindustrialbaseprograms/
A.J. Crispin, V. Rankov, Automated inspection of PCB components using a genetic algorithm template-matching approach. Int. J. Adv. Manuf. Technol. 35(3), 293–300 (2007). https://doi.org/10.1007/s00170-006-0730-0
A.M. Darwish, A.K. Jain, A rule based approach for visual pattern inspection. IEEE Trans. Pattern Anal. Mach. Intell. 10(1), 56–68 (1988). https://doi.org/10.1109/34.3867
N. Dave, V. Tambade, B. Pandhare, S. Saurav, PCB defect detection using image processing and embedded system. Int. Res. J. Eng. Technol. 3(5), 1897–1901 (2016)
A.R. de Mello, M.R. Stemmer, Inspecting surface mounted devices using k nearest neighbor and Multilayer Perceptron, in 2015 IEEE 24th International Symposium on Industrial Electronics (ISIE) (2015), pp. 950–955. https://doi.org/10.1109/ISIE.2015.7281599
J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, L. Fei-Fei, ImageNet: a large-scale hierarchical image database, in CVPR09 (2009)
N. Dhanachandra, K. Manglem, Y.J. Chanu, Image segmentation using K-means clustering algorithm and subtractive clustering algorithm. Proc. Comput. Sci. 54, 764–771 (2015). https://doi.org/10.1016/j.procs.2015.06.090
C. Fonseka, J. Jayasinghe, Feature extraction and template matching algorithm classification for PCB fiducial verification. J. Achiev. Mater. Manuf. Eng. 86(1), 14–32 (2018)
P. Ghosh, R.S. Chakraborty, Counterfeit IC detection by image texture analysis, in 2017 Euromicro Conference on Digital System Design (DSD), pp. 283–286 (2017)
P. Ghosh, R.S. Chakraborty, Recycled and remarked counterfeit integrated circuit detection by image-processing-based package texture and indent analysis. IEEE Trans. Ind. Inf. 15(4), 1966–1974 (2018)
P. Ghosh, D. Forte, D.L. Woodard, R.S. Chakraborty, Automated detection of pin defects on counterfeit microelectronics, in ISTFA 2018: Proceedings from the 44th International Symposium for Testing and Failure Analysis (2018), p. 57
P. Ghosh, A. Bhattacharya, D. Forte, R.S. Chakraborty, Automated defective pin detection for recycled microelectronics identification. J. Hardw. Syst. Secur. 3(3), 250–260 (2019)
P. Ghosh, U. Botero, F. Ganji, D. Woodard, R.S. Chakraborty, D. Forte, Automated detection and localization of counterfeit chip defects by texture analysis in infrared (IR) domain, in IEEE International Conference on Physical Assurance and Inspection of Electronics (PAINE) (2020)
R.C. Gonzalez, R.E. Woods, Digital Image Processing, 4th edn. (Pearson, London, 2017)
E. Guerra, J.R. Villalobos, A three-dimensional automated visual inspection system for SMT assembly. Comput. Ind. Eng. 40(1–2), 175–190 (2001)
U. Guin, D. DiMase, M. Tehranipoor, A comprehensive framework for counterfeit defect coverage analysis and detection assessment. J. Electron. Test. 30(1), 25–40 (2014)
U. Guin, K. Huang, D. DiMase, J.M. Carulli, M. Tehranipoor, Y. Makris, Counterfeit integrated circuits: a rising threat in the global semiconductor supply chain. Proc. IEEE 102(8), 1207–1228 (2014)
T. Jabid, M.H. Kabir, O. Chae, Local directional pattern (LDP) for face recognition, in 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE) (2010), pp. 329–330
M.J. Jianbo Shi, Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000). https://doi.org/10.1109/34.868688
B.C. Jiang, C.C. Wang, Y.N. Hsu, Machine vision and background remover-based approach for PCB solder joints inspection. Int. J. Prod. Res. 45(2), 451–464 (2007). https://doi.org/10.1080/00207540600607184
R. Johnson, The Navy Bought Fake Chinese Microchips That Could Have Disarmed U.S. Missiles. Business Insider (2011)
K.I. Laws, Rapid texture identification, in Image Processing for Missile Guidance, vol. 0238 (1980), pp. 376– 381
D.T. Lee, A computerized automatic inspection system for complex printed thick film patterns, in Applications of Electronic Imaging Systems, vol. 0143 (International Society for Optics and Photonics, Bellingham, WA, 1978), pp. 172–177. https://doi.org/10.1117/12.956563
W. Li, B. Esders, M. Breier, SMD segmentation for automated PCB recycling, in 2013 11th IEEE International Conference on Industrial Informatics (INDIN). ieeexplore.ieee.org (2013), pp. 65–70. https://doi.org/10.1109/INDIN.2013.6622859
D.U. Lim, Y.G. Kim, T.H. Park, SMD classification for automated optical inspection machine using convolution neural network, in 2019 Third IEEE International Conference on Robotic Computing (IRC) (2019), pp. 395–398
H. Lu, D. Mehta, O.D. Paradis, N. Asadizanjani, M. Tehranipoor, D.L. Woodard, FICS-PCB: a multi-modal image dataset for automated printed circuit board visual inspection (2020)
O.G. Luo Juan, A comparison of SIFT, PCA-SIFT and SURF. Int. J. Image Process 3(4), 143–152 (2009)
K. Mahmood, P.L. Carmona, S. Shahbazmohamadi, F. Pla, B. Javidi, Real-time automated counterfeit integrated circuit detection using X-ray microscopy. Appl. Opt. 54(13), D25–D32 (2015)
P.S. Malge, R.S. Nadaf, PCB defect detection, classification and localization using mathematical morphology and image processing tools. Int. J. Comput. Appl. Technol. 87(9), 40–45 (2014)
D. Mehta, H. Lu, O.D. Paradis, M. Azhagan, T. Rahman, P. Chawla, D.L. Woodard, M. Tehranipoor, N. Asadizanjani, The big hack explained: detection and prevention of PCB supply chain implants. ACM J. Emerg. Technol. Comput. Syst. https://doi.org/10.1145/3401980
M. Moganti, F. Ercal, C.H. Dagli, S. Tsunekawa, Automatic PCB inspection algorithms: a survey. Comput. Vis. Image Underst. 63(2), 287–313 (1996). https://doi.org/10.1006/cviu.1996.0020
T.S. Newman, A.K. Jain, A survey of automated visual inspection. Comput. Vis. Image Underst. 61(2), 231–262 (1995). https://doi.org/10.1006/cviu.1995.1017
A.N. Nowroz, K. Hu, F. Koushanfar, S. Reda, Novel techniques for high-sensitivity hardware Trojan detection using thermal and power maps. IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. 33(12), 1792–1805 (2014)
M. Piccardi, Background subtraction techniques: a review, in 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), vol. 4. ieeexplore.ieee.org (2004), pp. 3099–3104. https://doi.org/10.1109/ICSMC.2004.1400815
B.A. Procurement, Printed Circuit Board (PCB) Forecast Report for 2020. Tech. rep. (2019)
J. Li, J. Gu, Z. Huang, J. Wen, Application research of improved YOLO V3 algorithm in PCB electronic component detection. NATO Adv. Sci. Inst. Ser. E Appl. Sci. 9(18), 3750 (2019) https://doi.org/10.3390/app9183750
M. Reza, Z. Chen, D. Crandall, Deep neural network based detection and verification of microelectronic images. J. Hardw. Syst. Secur. 4, 44–54 (2020)
H. Rob, Counterfeit electronic component detection. https://www.aeri.com/counterfeit-electronic-component-detection/. Accessed 21 July 2020
K. Rosenfeld, R. Karri, Attacks and defenses for JTAG. IEEE Des. Test Comput. 27(1), 36–47 (2010)
S. Shahbazmohamadi, E.H. Jordan, Optimizing an SEM-based 3D surface imaging technique for recording bond coat surface geometry in thermal barrier coatings. Meas. Sci. Technol. 23(12), 125601 (2012)
S. Shahbazmohamadi, D. Forte, M. Tehranipoor, Advanced physical inspection methods for counterfeit detection, in Proceedings of International Symposium for Testing and Failure Analysis (ISFTA) (2014), pp. 55–64
S. Shahbazmohamadi, D. Forte, M. Tehranipoor, Advanced physical inspection methods for counterfeit IC detection, in ISTFA 2014: Conference Proceedings from the 40th International Symposium for Testing and Failure Analysis (2014), p. 55
Q. Shi, N. Vashistha, H. Lu, H. Shen, B. Tehranipoor, D.L. Woodard, N. Asadizanjani, Golden gates: a new hybrid approach for rapid hardware trojan detection using testing and imaging, in 2019 IEEE International Symposium on Hardware Oriented Security and Trust (HOST) (2019), pp. 61–71
Y. Shi, C.W. Ting, B.H. Gwee, Y. Ren, A highly efficient method for extracting FSMs from flattened gate-level netlist, in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (2010), pp. 2610–2613
Y.N. Sun, C.T. Tsai, A new model-based approach for industrial visual inspection. Pattern Recognit. 25(11), 1327–1336 (1992). https://doi.org/10.1016/0031-3203(92)90145-9
C. Szymanski, M.R. Stemmer, Automated PCB inspection in small series production based on SIFT algorithm, in 2015 IEEE 24th International Symposium on Industrial Electronics (ISIE). ieeexplore.ieee.org (2015), pp. 594–599. https://doi.org/10.1109/ISIE.2015.7281535
M. Tehranipoor, F. Koushanfar, A survey of hardware Trojan taxonomy and detection. IEEE Des. Test Comput. 27(1), 10–25 (2010)
N. Vashistha, H. Lu, Q. Shi, M.T. Rahman, H. Shen, D.L. Woodard, N. Asadizanjani, M. Tehranipoor, Trojan scanner: detecting hardware trojans with rapid SEM imaging combined with image processing and machine learning, in ISTFA 2018: Proceedings from the 44th International Symposium for Testing and Failure Analysis (2018), p. 256
N. Vashistha, M.T. Rahman, H. Shen, D.L. Woodard, N. Asadizanjani, M. Tehranipoor, Detecting hardware Trojans inserted by untrusted foundry using physical inspection and advanced image processing. J. Hardw. Syst. Secur. 2(4), 333–344 (2018)
T.J. Wagner, Hierarchical layout verification. IEEE Des. Test Comput. 2(1), 31–37 (1985)
D.Z. Wang, C.H. Wu, A. Ip, C.Y. Chan, D.W. Wang, Fast multi-template matching using a particle swarm optimization algorithm for PCB inspection, in Applications of Evolutionary Computing (Springer, Berlin, Heidelberg, 2008), pp. 365–370. https://doi.org/10.1007/978-3-540-78761-7_39
F. Wu, X. Zhang, Feature-extraction-based inspection algorithm for IC solder joints. IEEE Trans. Compon. Packaging Manuf. Technol. 1(5), 689–694 (2011). https://doi.org/10.1109/TCPMT.2011.2118208
H.H. Wu, X.M. Zhang, S.L. Hong, A visual inspection system for surface mounted components based on color features. in 2009 International Conference on Information and Automation (2009), pp. 571–576
H. Wu, G. Feng, H. Li, X. Zeng, Automated visual inspection of surface mounted chip components, in 2010 IEEE International Conference on Mechatronics and Automation (2010), pp. 1789–1794
L. Yang, X. Li, H. Li, Hardware Trojan detection method based on time feature of chip temperature, in 2020 10th Annual Computing and Communication Workshop and Conference (CCWC) (2020), pp. 1029–1032
C.H. Yeh, T.C. Shen, F.C. Wu, A case study: passive component inspection using a 1D wavelet transform. Int. J. Adv. Manuf. Technol. 22(11), 899–910 (2003). https://doi.org/10.1007/s00170-003-1608-z
S. Youn, Y. Lee, T. Park, Automatic classification of SMD packages using neural network, in 2014 IEEE/SICE International Symposium on System Integration (2014), pp. 790–795
Z. Zeng, Z.M. Li, Z. Zheng, Extracting PCB components based on color distribution of highlight areas. Comput. Sci. Inf. Syst. 7(1), 13–30 (2010)
B. Zhou, A. Aksoylar, K. Vigil, R. Adato, J. Tan, B. Goldberg, M.S. Ünlü, A. Joshi, Hardware Trojan detection using backside optical imaging. IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. (2020). https://doi.org/10.1109/TCAD.2020.2991680
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Lu, H., Capecci, D.E., Ghosh, P., Forte, D., Woodard, D.L. (2021). Computer Vision for Hardware Security. In: Tehranipoor, M. (eds) Emerging Topics in Hardware Security . Springer, Cham. https://doi.org/10.1007/978-3-030-64448-2_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-64448-2_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64447-5
Online ISBN: 978-3-030-64448-2
eBook Packages: EngineeringEngineering (R0)