1 Introduction

The recent globalization in the semiconductor industry has reduced the overall cost of electronic devices; however, it has introduced hardware security threats like IC cloning, IC piracy, hardware trojan, and IC counterfeit [1]. A major source leading to counterfeit ICs is outsourcing them to untrusted parties in manufacturing and packaging without standard analysis. The counterfeit chips impact the reliability and performance of the systems as the aged chips’ performance deteriorates over a period of time causing system failures [2]. Access to recycled ICs may lead to successfully reverse engineer IC design leading to critical data leakage [3]. Access to counterfeit and recycled chips has challenged the protection of IP rights as they can be cloned and modified [4]. Thus, to ensure the authenticity of the component, we propose counterfeit detection models to ensure authenticity of the component and catching the counterfeit ICs early in the supply chain process.

The reliability and security of electronic systems depend heavily on the successful identification of counterfeit integrated circuits (ICs). In this process, both electrical and physical inspection are crucial [2]. Physical examination entails closely examining the exterior characteristics, markings, and packaging of ICs to find any irregularities or discrepancies that would point to counterfeiting [5]. This is important since counterfeiters frequently try to look just like authentic ICs. Conversely, electrical inspection entails using different testing techniques, like parametric measurements and functional testing, to evaluate the electrical properties and performance of the integrated circuit [4]. This assists in identifying differences between the observed and expected behavior of the IC. Because it covers many aspects of counterfeiting, integrating both physical and electrical examinations improves the overall detection accuracy. Manual inspection techniques, on the other hand, are time-consuming and prone to human error, making them inefficient for the increasing complexity and volume of ICs in modern electronics. A preferable option is automated physical inspection, which makes use of advanced imaging technologies, machine learning algorithms, and computer vision. It enables the speedy and exact examination of huge batches of ICs, allowing for the detection of small anomalies that would otherwise go undetected in manual inspections. Automated systems can detect anomalies in packing, markings, and other physical properties with high accuracy, lowering the danger of false negatives or positives and expediting the counterfeit detection process [6,7,8].

Detecting counterfeit ICs is a challenging task due to the sophisticated techniques used by counterfeiters to replicate authentic ICs. Machine learning has emerged as a powerful tool for counterfeit IC detection, leveraging its ability to analyze complex patterns and extract distinguishing features. Several studies have investigated the application of machine learning algorithms, particularly deep learning, for accurate and efficient counterfeit IC detection. These approaches harness the power of neural networks to analyze various aspects of ICs, including visual features, electrical behavior, and physical characteristics. The integration of machine learning techniques holds great promise for developing robust and automated systems capable of ensuring the integrity and authenticity of ICs in a wide range of applications.

In this paper, we evaluate transfer learning methods that can be built using models trained on large datasets and transfer the knowledge learned on a customized labeled database of authentic and counterfeit images. Transfer learning is a specialized technique in machine learning [9] that can be used in different fields of classification. The initial form of the model is trained on a larger dataset not specific to our field of study. The knowledge learned from the model can be used to train further on specific authentic and counterfeit images. Contrary to transfer learning, generative autoencoder networks do not require large datasets to train and come under the umbrella of unsupervised learning. They are a way to learn deep representations without using extensively annotated training data.

To summarize, this paper makes the following contributions:

  1. 1.

    The development of a novel autoencoding architecture to determine counterfeit ICs versus authentic ICs

  2. 2.

    A comparative analysis of the autoencoding architecture versus different transfer learning models spanning various adavantages

  3. 3.

    A comprehensive analysis of the results from the experiment employing four different classification metrics (precision, recall, F1 score, and accuracy) to understand the effectiveness of the proposed model under testing dataset

  4. 4.

    A thorough model resource analysis employing three different metrics (FLOPs, MACs, and trainable parameters) which measure the computational complexity and size of the model

This paper is organized as follows: Section 2 describes the related work; Section 3 covers the problem statement. Section 4 describes the proposed methodology; Section 5 discusses the experimental results. Section 6 presents the security analysis and Section 7 discusses the conclusion.

2 Related Work

The physical inspection of integrated circuits (ICs) and printed circuit boards (PCBs) is a time-consuming process that involves the use of image processing, computer vision, and machine learning techniques. This approach, which takes inspiration from established practices in industrial quality control and production automation, involves systematically inspecting ICs and PCBs for defects and irregularities [10]. The importance of physical examination lies in its ability to conduct non-destructive analysis, making it crucial in the field of counterfeit detection. By preserving the structural integrity of genuine ICs and swiftly identifying abnormalities that indicate counterfeit components, this method plays a vital role in ensuring the reliability and performance of electronic systems [11]. In addition, physical examination reduces the likelihood of human error linked to manual inspections performed by subject matter experts (SMEs) using advanced technology and automated procedures. Its significance in the domain of counterfeit detection is underscored by its capacity to identify subtle alterations and flaws that may go unnoticed by traditional inspection techniques, thereby enhancing the overall dependability and security of electronic equipment in a diverse range of industrial applications [5, 12].

Counterfeit detection using physical inspection involves study of physical properties [5]. X-ray imaging techniques are utilized to inspect concealed packaging surfaces and identify features of the components [13]. For detecting defects in integrated circuits (IC), X-ray microscopy combined with image processing algorithms is applied to extract features from images. Surface texture–based image processing, which involves texture analysis, is employed for images with lower resolution [14]. To detect feature extraction and establish a foundation for real defect classification and recognition, image enhancement techniques such as histogram equalization and transformations are used [15]. Segmentation-based models [16] utilize clustering algorithms for feature classification with semi-automated frameworks. However, despite the effectiveness of image processing techniques in identifying counterfeit ICs, their accuracy is limited due to resource constraints and dependence on the defects being identified.

Machine learning techniques for counterfeit detection include supervised and unsupervised learning. Supervised includes transfer learning and unsupervised in the form of GANs [6, 17]. In supervised learning, the training process includes providing labeled datasets. Integration of physical inspection can be used to increase accuracy. During the training phase, the images are pre-processed using various techniques and error correction schemes, while the authenticity of the components is determined based on the trained model during the testing phase. Ahmadi et al. [18] proposed the use of X-ray 3D imaging to detect die-face delamination and trained a logistic regression model for detection. Aramoon and Qu [19] mention surveys and list those from these papers, and briefly reviewed different emerging machine learning–based counterfeit detection techniques with various algorithms. Other techniques include EMFORCED [20] that utilizes a reference-free and reference-inclusive classification method based on electromagnetic emissions. The EM emissions from clock distribution mechanism are used as fingerprints to detect counterfeit ICs. Furthermore, material based characterization to train the classification models, can provide interpretability and hardware assurance capabilities Xi et al. [7]. These advanced techniques are limited with the requirement with the limited availability of data for the training process, which leads to less accurate models and degraded classification rates. The following machine learning techniques are discussed in the following subsections.

2.1 Transfer Learning

Transfer learning has emerged as a powerful technique in the field of machine learning, allowing models to leverage knowledge from one domain and apply to other domains. Various studies have explored different aspects of transfer learning, including its theoretical foundations, practical applications, and performance evaluation. Pan and Yang (2010) presented a comprehensive survey on transfer learning, categorizing it into different scenarios such as supervised, unsupervised, and reinforcement learning [21]. The survey highlights the key challenges and opportunities of transfer learning. Recent research has focused on enhancing transfer learning methods through deep learning architectures. A seminal work by Yosinski et al. demonstrated the effectiveness of transfer learning by fine-tuning deep neural networks pre-trained on large-scale image datasets [22]. The study shows that pre-training on a large dataset, such as ImageNet, followed by fine-tuning on a specific task, significantly improves the model’s performance and reduces training time. Transfer learning is extensively explored using deep learning architectures, notably, the VGG16, VGG19, and ResNet [23]. The VGG16 and VGG19 models are deep convolutional neural networks with 16 and 19 layers, respectively. These models are pre-trained on the ImageNet dataset, consisting of millions of labeled images, and have shown significant improvement in performance on various computer vision tasks, including object recognition and image classification. Similarly, the ResNet model, known for its residual connections, has achieved state-of-the-art results on ImageNet and other benchmark datasets [24]. Its ability to effectively train very deep networks has made it a popular choice for transfer learning in computer vision tasks. InceptionNet V3, also known as Inception-v3, is a deep convolutional neural network architecture that has made significant contributions to the field of computer vision [25]. It is renowned for its remarkable performance in image classification tasks. The key innovation of the Inception architecture is its use of “Inception modules,” which are convolutional modules with multiple filter sizes and operations, allowing the network to capture features at various scales. This architectural design promotes better utilization of computational resources while maintaining high accuracy. Similarly, EfficientNet B1 is a member of the EfficientNet family of neural network architectures, which have gained attention for their impressive computational efficiency and competitive accuracy [26]. These models are designed to optimize both model size and computational resources, making them highly efficient in various deep learning applications.

2.2 Autoencoders

Autoencoders are a type of neural network architecture primarily used for unsupervised learning and dimensionality reduction. They consist of an encoder and a decoder. The encoder compresses the input data into a lower-dimensional latent space, and the decoder attempts to reconstruct the original input from this compressed representation. Autoencoders are commonly used for tasks like data denoising, image compression, and anomaly detection.

In the seminal work [27] in this area of autoencoders, the authors propose a deep autoencoder model that learns a hierarchical representation of the input data. The model trained on the MNIST dataset outperforms traditional shallow models. Another significant advancement in autoencoder neural networks is the use of convolutional neural networks (CNNs) as the encoder and decoder. Krizhevsky and Hinton [28] propose a deep convolutional autoencoder that learns to reconstruct input images from their lower-dimensional representations. The authors also showed that these models can be used for feature learning tasks such as object recognition and detection. Variational autoencoders are generative models that learn a probabilistic distribution over the input data that can be used for image generation tasks [29]. VQ-VAE [30] is an improved variation of VAE called vector quantized VAE (VQ-VAE) that produces high-quality and diverse images by using a codebook to discretize the latent space. Another recent improvement in autoencoder neural networks is the use of attention mechanisms. Sutskever et al. [31] proposes a data distribution technique named “DRAW” that uses an attention mechanism to selectively focus on different parts of the image during the reconstruction process. This model shows high-quality reconstructions on the MNIST dataset. Autoencoders are also shown to perform well in anomaly detection tasks such as [32] where an LSTM autoencoder was used as an unsupervised learning framework to detect probable anomalies in a dataset involving average power consumption data in a microgrid.

3 Problem Statement

The problem at hand revolves around the detection of counterfeit integrated circuits (ICs) from a dataset of images that includes both authentic and counterfeit ICs. Counterfeit ICs pose a significant threat to various industries, including electronics and aerospace, as they can lead to system failures, security breaches, and financial losses. Given the limited availability of images, the use of autoencoder networks emerges as a promising approach. Autoencoders excel at learning from small datasets by encoding images into their latent representations and then decoding them back to their original form. The encoder portion of the autoencoders can then be utilized for classifying between authentic and counterfeit image classes based on a set threshold. The synthetic data augmentation used here should induce variations in the dataset to ensure minimal bias in the model predictions while accurately detecting the counterfeit ICs and reducing the computational overhead and cost (measured here using FLOPs and MACs) of the transfer learning models. The models should also cause a significant reduction in the model size so that they can be deployed in resource-constrained environments as well.

4 Proposed Methodology

The proposed methodology section describes the data and the models that will be trained on the data. To keep the results consistent, the testing data is taken out from the overall dataset and will be used for evaluating all the models. This ensures that the results and metrics are representative of the same dataset. The following sections mention the statistics about image data used for the purposes of this paper and the pre-processing techniques and augmentations applied on them. Further sections will delve into the details of the models implemented and their respective results.

4.1 Data and Pre-processing

The dataset consists of 196 high-resolution images (2560 \(\times\) 1920) acquired from a Zeiss STEMI 508 Stereo Microscope. The images belong to two classes: authentic and counterfeit. There are 145 samples in the authentic class and 51 samples in the counterfeit class. As mentioned above, a test dataset of 41 images was taken out at random from the original dataset which had a distribution of 25 samples in the authentic class and 16 samples in the counterfeit class. This ensures \(\approx\) 80% split for training data and \(\approx\) 20% split for test data. The meta data also include additional information about vendors and manufacturers along with the number of pins (e.g., 8, 16, 64) and the packaging types such as dual inline package (DIP) and quad flat package (QFP).

The 155 samples in the training set followed a distribution of 120 authentic samples and 35 counterfeit samples. The learning process of an autoencoder model involves training the model on the majority class (authentic class in this case) and test the model on a minority class (counterfeit class in this case). The intuition behind that is the fact that an autoencoder model tries to learn to reconstruct an input image. The reconstruction error between the reconstructed image and the input image for the majority class should be different than the reconstruction error evaluated for the minority class such that a threshold value of the reconstruction error can be set which can determine with certain degree of confidence that a particular image belongs to the authentic class or the counterfeit class. To this end, the 120 authentic samples form the training dataset of which 90 samples were used for training and 30 were used for model validation.

In contrast to this, the learning process for any transfer learning model involves training the model on all of the available classes and perform predictions on the test data. To this end, for the transfer learning models, the training data involved 155 images in total of which 100 were used for training and 55 were used for validation. This number is different than the one used for training the autoencoder because the 35 images from the training set belonging to the counterfeit class were also considered in the training process for the transfer learning models.

After the creation of the training and test dataset, an autoencoder model was developed (AutoDetect) and would be compared against four different transfer learning models. All the models would be trained twice, once on pre-processed data and the second time on pre-processed and augmented data. Then, the models would be evaluated comprehensively based on a set of classification metrics as well as considering the efficiency and complexity of the model.

For pre-processing the images, all of them were scaled down to 224 \(\times\) 224 pixels and were applied with samplewise centering and samplewise standardization techniques. These techniques are used to modify the pixel values of each image, making them suitable for training machine learning models, particularly deep neural networks, which are sensitive to the scale and distribution of input data. Samplewise centering involves subtracting the mean pixel value of each image from all the pixels within that image. This helps remove the overall brightness or contrast bias from individual images. By centering each image around its mean pixel value, they can have a similar overall intensity level. Samplewise standardization for images involves scaling the pixel values within each image so that they have a mean of zero and a standard deviation of one. This is done by subtracting the mean pixel value of the image from each pixel and then dividing by the standard deviation of the image’s pixel values. It ensures that the pixel values within each image have a consistent scale. This can be crucial when working with machine learning models, especially deep neural networks, because models tend to perform better when input data has similar scales. It can lead to improved convergence during training and better model performance.

Additionally, the dataset contains supplementary metadata such as information about vendors, manufacturers, the number of pins, and packaging types. This ensures that the dataset provides a diverse representation of IC packages. In order to address the inherent variations within a single batch, we applied data augmentation techniques during the pre-processing stage which will be discussed in more detail in the following subsection. These techniques included image standardization and centering. However, it is important to emphasize that these variations within a batch contribute to the overall diversity of the dataset, thus enabling the model to effectively generalize.

4.2 Image Augmentation

Image augmentations are a crucial technique in computer vision and image processing, particularly in the context of training deep learning models for tasks like image classification, object detection, segmentation, and more. They involve applying various transformations or modifications to the original images in the dataset to create new, slightly modified versions of those images. These augmented images are then used alongside the original images during training. To keep the training process as consistent as possible, the different augmentations applied to the images were kept the same across all models. One of the primary benefits of image augmentations is that they effectively increase the size of the training dataset. Augmentations introduce variations in the training data, such as changes in scale, rotation, or lighting conditions. This forces the model to learn more robust features and patterns, enabling it to generalize better to unseen or real-world scenarios. In classification tasks, class imbalance can be a significant challenge. Augmentations can help address this issue by creating additional examples of minority classes. This balances the class distribution and ensures that the model does not become biased towards the majority class.

The specific set of augmentations used on this dataset were as follows:

Image Rotation: Rotation augmentation introduces random rotations to the input images within a specified range. This augmentation mimics variations in object or camera orientation, promoting the model’s ability to recognize objects from different angles.

Width Shift Range: Width shift augmentation horizontally shifts the image by a fraction of its width. This augmentation simulates changes in the position of objects within the frame encouraging the model to learn features that are invariant to horizontal translation.

Height Shift Range: Height shift augmentation vertically shifts the image by a fraction of its height. This augmentation models variations in object position along the vertical axis.

Shear Range: Shear augmentation applies a shearing transformation to the image, which distorts it along one axis. Shearing mimics deformations that can occur when objects are viewed from oblique angles.

Zoom Range: Zoom augmentation randomly zooms in or out of the image within the specified range. Zoom augmentation simulates changes in object scale or focal length, helping the model become robust to variations in object size.

Horizontal Flipping: Horizontal flip augmentation mirrors the image horizontally with a 50% probability. This augmentation models the possibility of objects being oriented in the opposite direction.

4.3 Transfer Learning

Transfer learning has emerged as a transformative paradigm in the realm of deep learning, particularly in the domain of computer vision. It begins with pre-trained neural network models, which have been trained on extensive datasets, such as ImageNet. These models serve as repositories of knowledge, encapsulating learned representations of visual features. At its core, transfer learning hinges on the notion of transferring the knowledge embedded within pre-trained models to new and diverse tasks as shown in Fig. 1. This knowledge encompasses hierarchical features, patterns, and abstractions acquired during the initial training process. In the context of binary image classification, transfer learning exploits the knowledge embedded within a pre-trained model to enhance the model’s performance on a specific binary classification task.

Fig. 1
figure 1

Transfer learning methodology

Full size image

By leveraging pre-trained models, we substantially reduce the computational resources required for training. Transfer learning is particularly beneficial when the target dataset is limited such as the one used in this paper. The knowledge transferred from the pre-trained model often includes robust, high-level features that are beneficial for a wide range of tasks. This results in improved classification performance on the binary task. Specifically four transfer learning models were used in this research for comparison with the autoencoder model. These were based on their performance on the ImageNet dataset as well as their size and number of parameters. The goal of this paper is to evaluate the autoencoder against the transfer learning approaches not only on the basis of accuracy but also on the basis of efficiency. Thus, the transfer learning models chosen were in decreasing order of size and number of parameters while still maintaining high accuracy on the ImageNet dataset as summarized in Table 1.

Table 1 Transfer learning models and their performance on ImageNet dataset
Full size table

These pre-trained models are trained on the ImageNet dataset consisting of millions of images belonging to 1000 different classes for a number of days to achieve approximately 90 to 95% accuracy on the test set of the ImageNet data. Compared to the millions of images, our dataset is considerably small and hence, we chose to freeze the weights of the pre-trained models retaining the weights as learned from the ImageNet dataset. These models are adapted for a binary image classification task (authentic versus counterfeit) in this case by removing the last layer which contains 1000 neurons corresponding to the 1000 classes in the ImageNet dataset and replacing them by an output layer with 1 neuron and applying a sigmoid activation function for predicting probablities of the binary output classes. The probabilities greater than 0.5 correspond to the counterfeit class while the ones less than 0.5 correspond to the authentic class. The Adam optimizer, short for adaptive moment estimation, is a widely used optimization algorithm for training neural networks. The BCE loss function outputs probabilities for binary classification tasks. Minimizing this loss encourages the predicted probabilities to align with the true labels, resulting in improved model performance for binary classification problems. The training process for all of the selected transfer learning models is summarized in Algorithm 1.

Algorithm 1
figure a

Transfer Learning Training Process

Full size image

The following sections delve into the architectural details and key ideas regarding the specific transfer learning models used in this paper.

4.3.1 VGG16 Model

The VGG16 (Visual Geometry Group 16) model is a convolutional neural network architecture that was developed by the Visual Geometry Group at the University of Oxford. VGG16 is a part of the VGG family of models, which includes various architectures with different depths, denoted by numbers like VGG16 and VGG19. The VGG16 model has a very simple and uniform architecture consisting of 16 weight layers, which include 13 convolutional layers and 3 fully connected layers. The key characteristics of the VGG16 architecture are as follows:

  • Convolutional Layers: The first two layers are 2D convolutional layers with a small 3 \(\times\) 3 receptive field. They use a stride of 1 and a padding of 1. Following these initial layers, there are 11 more convolutional layers, each with a 3 \(\times\) 3 receptive field and a stride of 1. These convolutional layers are followed by max-pooling layers with a 2 \(\times\) 2 window and a stride of 2 after every two convolutional layers.

  • Fully Connected Layers: The convolutional layers are followed by three fully connected layers. The first two fully connected layers have 4096 neurons each, and the final fully connected layer has 1000 neurons for classification. The output of the final layer is typically passed through a softmax activation function to produce class probabilities.

  • Activation Function: Rectified linear unit (ReLU) activation functions are used after each convolutional and fully connected layer, except for the final output layer.

  • Normalization: Batch normalization is applied after each convolutional and fully connected layer to improve training stability and speed up convergence.

  • Dropout: Dropout is used as a regularization technique in the fully connected layers to prevent overfitting.

4.3.2 ResNet Model

The ResNet (residual network) model is a groundbreaking deep convolutional neural network architecture that revolutionized the field of computer vision. Developed by Microsoft Research, ResNet introduced a novel architectural concept that addressed the vanishing gradient problem and enabled the training of extremely deep neural networks.

  • Residual Blocks: The core innovation of ResNet is the use of residual blocks. A residual block consists of skip connections, also known as shortcut connections, that allow the network to skip one or more layers. These connections enable the gradient to flow more easily during training, alleviating the vanishing gradient problem. Each residual block typically contains two convolutional layers with batch normalization and ReLU activations.

  • Identity Shortcut: The simplest form of a residual block includes an identity shortcut connection, where the input tensor is added element-wise to the output tensor of the block. This identity shortcut helps maintain the information from earlier layers and facilitates the training of very deep networks.

  • Bottleneck Architectures: In practice, ResNet often employs bottleneck architectures in its residual blocks. These bottleneck blocks consist of three layers: 1 \(\times\) 1 convolution, 3 \(\times\) 3 convolution, and another 1 \(\times\) 1 convolution. These layers reduce the computational complexity while preserving representational power, making it possible to train even deeper networks efficiently.

  • Variants: ResNet comes in several variants, including ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152, which differ in terms of depth. Deeper variants are capable of capturing more complex features and patterns.

  • Pre-trained Models: Pre-trained ResNet models are available, trained on large-scale image datasets such as ImageNet. These pre-trained models serve as powerful feature extractors and can be fine-tuned for various computer vision tasks.

ResNet’s innovative use of residual blocks and skip connections has paved the way for training extremely deep neural networks with hundreds of layers. While ResNet has had a significant impact on computer vision, its architecture remains conceptually simpler than many other deep networks, making it a valuable asset for both research and practical applications.

4.3.3 Inceptionv3 Model

The InceptionV3 model, part of the Inception family of neural networks, represents a significant advancement in convolutional neural network (CNN) architectures. Developed by Google, InceptionV3 is known for its efficiency in terms of model parameters while maintaining high accuracy.

  • Stem and Feature Extraction: InceptionV3 begins with a stem module that performs initial convolution and pooling operations to extract basic features from input images. It then proceeds through multiple inception modules, each containing a series of convolutional and pooling layers with various kernel sizes.

  • Inception Modules: The hallmark of Inception models is the use of inception modules, which consist of convolutional layers with 1 \(\times\) 1, 3 \(\times\) 3, and 5 \(\times\) 5 kernels, as well as max-pooling layers. These operations are performed in parallel, and their outputs are concatenated. This parallel processing allows the model to capture features at different receptive field sizes efficiently.

  • Auxiliary Classifiers: InceptionV3 includes auxiliary classifiers at intermediate layers during training to encourage gradient flow and prevent the vanishing gradient problem. These classifiers aid in training and are usually discarded during inference.

  • Global Average Pooling: Instead of using fully connected layers at the end, InceptionV3 employs global average pooling, which reduces the spatial dimensions of the feature maps to a single vector per feature map. This leads to a significant reduction in the number of parameters.

  • Final Classification: The output of global average pooling is fed into a fully connected layer for final classification. The model typically ends with a softmax activation function for multiclass classification tasks.

Thus, by efficiently capturing features at various scales and reducing overfitting, InceptionV3 achieves competitive performance with fewer parameters compared to VGG models. This efficiency is especially valuable in environments, such as mobile and embedded applications where resources are limited.

4.3.4 EfficientNetB1 Model

The EfficientNetB1 model is part of the EfficientNet family of neural networks, which are known for their remarkable efficiency and high performance in various computer vision tasks. Also developed by Google, EfficientNet models are designed to achieve state-of-the-art results while requiring fewer parameters compared to traditional architectures.

  • Stem and Feature Extraction: EfficientNetB1 starts with a stem module that performs initial convolution and pooling operations to extract basic features from input images. It then proceeds through multiple blocks, each containing several inverted residual blocks.

  • Inverted Residual Blocks: The key building block of EfficientNet is the inverted residual block, which is designed to minimize computation and memory usage while maintaining expressive power. These blocks consist of depthwise separable convolutions, which are computationally efficient, followed by pointwise convolutions to expand the number of channels.

  • Scaling Factors: EfficientNet models use scaling factors (\(\phi\)) to control the depth, width, and resolution of the network. These factors are typically chosen to balance model complexity and performance. EfficientNetB1 has a baseline scaling factor of 1.

  • Efficient Use of Resources: The architecture efficiently utilizes resources by increasing depth, width, and resolution in a coordinated manner. This results in a network that achieves high accuracy with fewer parameters.

  • Global Average Pooling: Like Inception models, EfficientNet also employs global average pooling at the end of the network to reduce spatial dimensions and parameters before the final classification layer.

  • Final Classification: The output of global average pooling is connected to a fully connected layer for the final classification. The model typically ends with a softmax activation function for multiclass classification tasks.

EfficientNet models achieve high performance while using fewer parameters, making them suitable for deployment on resource-constrained devices and applications. Its balance between model complexity and performance makes it a valuable choice for a wide range of computer vision applications.

4.4 Autoencoder Learning

Autoencoder networks are a class of artificial neural networks that are widely used for unsupervised learning tasks, particularly in the field of dimensionality reduction and data compression. The fundamental structure of an autoencoder consists of two main components: an encoder and a decoder as shown in Fig. 2.

Fig. 2
figure 2

Autoencoder model architecture

Full size image

The encoder takes the input data and transforms it into a lower-dimensional representation, often referred to as the latent space or code. This transformation process involves a series of mathematical operations, such as linear transformations and non-linear activation functions. The encoder’s objective is to capture the most important features and patterns present in the input data while reducing its dimensionality. The resulting compressed representation, or latent code, typically has a lower dimensionality compared to the original input data. This compressed representation serves as a condensed and informative representation of the input data, capturing its essential characteristics. The decoder component of the autoencoder takes the latent code as input and aims to reconstruct the original input data from this compressed representation. Similar to the encoder, the decoder consists of layers that perform mathematical operations, including activation functions and linear transformations, to transform the latent code back into the original input space.

Autoencoders are unsupervised models for feature extraction and representation learning, uncovering meaningful features from raw data without manual engineering. Unlike pre-trained models, they adapt specifically to the data, enhancing relevance. Used for data transformation and denoising, they learn compressed representations to remove noise and extract domain-specific features. Our proposed autoencoder architecture for classifying authentic and counterfeit images outperforms transfer learning models in classification metrics while reducing model parameters and computational operations. During training, autoencoders minimize the difference between input and reconstructed output via optimization algorithms, gradually learning to encode data into a lower-dimensional latent space for efficient storage, compression, and anomaly detection.

The encoder part of the autoencoder is responsible for reducing the dimensionality of the input data while retaining its essential features. The essential components of an encoder are as follows:

  • Convolutional Layers: The encoder starts with a series of convolutional layers. Each convolutional layer applies a set of learnable filters to the input data, producing feature maps. The number of filters in each layer determines the depth of the feature maps.

  • Activation Function (ReLU): After each convolutional layer, the rectified linear unit (ReLU) activation function is applied element-wise. Mathematically, this is represented as:

    $$\begin{aligned} \text {ReLU}(x) = \max (0, x) \end{aligned}$$

    where \(x\) is the input to the activation function.

  • Max-Pooling Layers: Max-pooling layers reduce the spatial dimensions of the feature maps by selecting the maximum value from a local region. This downsampling helps in capturing the most important features. Mathematically, max-pooling with a 2 \(\times\) 2 window can be expressed as:

    $$\begin{aligned} \text {MaxPooling}(x) = \max (x_{i,j}), \quad i, j \text { within the local region} \end{aligned}$$

The decoder part of the autoencoder aims to reconstruct the original input data from the compressed representation obtained by the encoder. It is responsible for upsampling the features back to the original dimensions. The important layers of a decoder are mentioned as follows:

  • Convolutional Layers: The decoder mirrors the encoder’s structure with convolutional layers. These layers gradually increase the spatial dimensions of the feature maps.

  • UpSampling Layers: UpSampling layers increase the spatial dimensions by replicating the values in a local region. This upsampling operation is often performed with a 2 \(\times\) 2 window, effectively doubling the spatial dimensions.

  • Final Layer: The final convolutional layer in the decoder outputs the reconstructed image. The activation function used here is the sigmoid activation function, which ensures that the pixel values are in the range [0, 1].

The autoencoder is trained to minimize the mean squared error (MSE) loss between the input data and the reconstructed output data and optimized using the Adam optimizer. Mathematically, the MSE loss is defined as:

$$\begin{aligned} \text {MSE}(Y_{\text {true}}, Y_{\text {pred}}) = \frac{1}{N} \sum _{i=1}^{N} (Y_{\text {true}}^i - Y_{\text {pred}}^i)^2 \end{aligned}$$

where \(Y_{\text {true}}\) represents the true input data, \(Y_{\text {pred}}\) is the reconstructed output, and \(N\) is the number of data points.

The MSE loss quantifies the discrepancy between the input data and the reconstructed data. Using MSE as the reconstruction error encourages the autoencoder to learn a representation that minimizes the reconstruction error, effectively capturing the essential features of the data. When using an autoencoder for binary classification, the MSE loss is seen as a measure of how well the autoencoder can reconstruct the input data. Low MSE indicates that the autoencoder is good at preserving the key features of the input samples. After training the autoencoder, it can be used for various tasks, including anomaly detection in binary classification. Anomaly detection involves distinguishing between normal (authentic) and anomalous (counterfeit) samples based on the autoencoder’s reconstruction error. This process is defined in Algorithm 2.

Algorithm 2
figure b

Autoencoder Training and Thresholding for Authentic vs. Counterfeit Images

Full size image

Now that we have the threshold T set, we can use it to classify new images from a test dataset as either authentic or counterfeit. For each test image \(\textbf{I}_{\text {Test}}\) in the test dataset, we perform the following steps:

  1. 1.

    Encode Test Image: We start by encoding the test image. This is achieved by passing \(\textbf{I}_{\text {Test}}\) through the same encoder network used during training. The result is a compressed representation of the test image, which we denote as \(\textbf{h}_{\text {Test}}\).

  2. 2.

    Reconstruct Test Image: Having obtained the encoded representation, we proceed to reconstruct the test image. To accomplish this, we input \(\textbf{h}_{\text {Test}}\) into the decoder network. The decoder network is responsible for generating a reconstruction of the test image, represented as \(\mathbf {I'}_{\text {Test}}\).

  3. 3.

    Calculate Reconstruction Error: To assess the quality of the reconstruction, we calculate the reconstruction error, denoted as \(\mathcal {E}_{\text {Test}}\). This error is computed using the same mean squared error (MSE) loss function as employed during training. It quantifies the disparity between the original test image \(\textbf{I}_{\text {Test}}\) and its reconstruction \(\mathbf {I'}_{\text {Test}}\).

  4. 4.

    Classification Decision: Based on the calculated reconstruction error \(\mathcal {E}_{\text {Test}}\), we make a classification decision for the test image: - If \(\mathcal {E}_{\text {Test}}\) is less than the predefined threshold T, we classify the test image as authentic. This decision arises because the reconstruction error falls within the typical range observed during training. - If \(\mathcal {E}_{\text {Test}}\) is greater than or equal to the threshold T, we classify the test image as counterfeit. This classification results from the reconstruction error surpassing the threshold, indicating a significant deviation from the typical characteristics of authentic images.

With the results from the autoencoder model on the test data, we compare it with the other transfer learning methods on different classification metrics as explained in the following section.

5 Experimental Results

In this section, we present the experimental results of our study, focusing on the data used, the evaluation metrics employed for model accuracy comparison, and other metrics for model efficiency utilization.

The computational resources utilized for running training and inference included the Google Colaboratory platform and the Tesla K80 GPU involving 16GB memory. The overall model training times on this platform is detailed in Table 2.

Table 2 Training details of models with and without data augmentation
Full size table

The loss convergence plots after training the proposed autoencoder model in the “without data augmentation” and “with data augmentation” configuration are shown in Fig. 3.

Fig. 3
figure 3

Loss convergence plots for AutoDetect without and with data augmentation configuration

Full size image

t-SNE (t-distributed stochastic neighbor embedding) is a dimensionality reduction technique commonly used for visualizing high-dimensional data in lower-dimensional space, typically two or three dimensions. It works by modeling the relationships between data points in the high-dimensional space and then mapping them to a lower-dimensional space while preserving the pairwise similarities as much as possible. Figure 4 shows the t-SNE components of authentic and counterfeit images visualized in two dimensions. These t-SNE components are computed using the encoder portion of the autoencoder model visualizing the latent representations learned by the autoencoder. There is some overlap between authentic and counterfeit Images when considering the original images; however, there exists clear clusters when using the augmented image dataset. Thus, the data augmentations mentioned in this paper also help in effectively separating the authentic and counterfeit images consequently increasing the classification accuracy of the proposed autoencoder model with data augmentation as shown in the following results.

Fig. 4
figure 4

T-SNE visualization of components between original and augmented authentic and counterfeit images

Full size image

To evaluate the performance of the different models, we rely on various classification metrics, including precision, recall, and the F1 score. These metrics provide valuable insights into the model’s ability to make accurate predictions.

Precision: Precision measures the accuracy of positive predictions made by the model. It is the ratio of true positives (correctly identified authentic samples) to the total number of samples predicted as authentic.

$$\begin{aligned} \text {Precision} = \frac{\text {True Positives}}{\text {True Positives} + \text {False Positives}} \end{aligned}$$

Recall (Sensitivity): Recall quantifies the model’s ability to correctly identify all positive instances. It is the ratio of true positives to the total number of actual authentic samples.

$$\begin{aligned} \text {Recall} = \frac{\text {True Positives}}{\text {True Positives} + \text {False Negatives}} \end{aligned}$$

F1 score: The F1 score is the harmonic mean of precision and recall. It provides a balance between Precision and Recall and is particularly valuable when dealing with imbalanced datasets or when both false positives and false negatives need to be minimized.

$$\begin{aligned} \text {F1 Score} = 2 \cdot \frac{\text {Precision} \cdot \text {Recall}}{\text {Precision} + \text {Recall}} \end{aligned}$$

The F1 score is important because it considers both false positives (type I error) and false negatives (type II error). Type I errors occur when counterfeit samples are incorrectly classified as authentic, while type II errors occur when authentic samples are incorrectly classified as counterfeit. In counterfeit detection, striking a balance between these errors is crucial. The importance of minimizing type I and type II errors depends on the specific application and its consequences. Minimizing type I errors (false positives) is crucial when the cost or consequences of incorrectly classifying counterfeit products as authentic are high. This ensures that counterfeit products do not pose a significant threat or danger. In such cases, it is essential to identify all potential counterfeits, even if it means accepting some false positives, while minimizing type II errors (false negatives) ensures that genuine products are not rejected.

Along with the abovementioned classification metrics, we employ floating-point operations (FLOPs), multiply-accumulate operations (MACs), and the number of trainable model parameters as metrics to assess the efficiency of the different models. These metrics quantify the computational complexity of a model and are used to evaluate its speed and resource requirements.

FLOPs (floating-point operations): FLOPs measure the total number of floating-point operations (e.g., additions and multiplications) performed during inference. Lower FLOPs indicate a more computationally efficient model.

MACs (multiply-accumulate operations): MACs count the number of multiply-accumulate operations, which are fundamental in many neural network operations. Efficient models have lower MAC counts.

Model parameters: The number of model parameters is another crucial factor in evaluating the efficiency of a model. Model parameters are the weights and biases learned by the neural network during training. A larger number of parameters typically leads to a larger model size, which can require more storage space and memory during inference. Smaller models with fewer parameters are more memory-efficient and are preferable for resource-constrained environments. Extremely large models with an excessive number of parameters may be prone to overfitting, especially when the training dataset is limited. Smaller, more efficient models tend to generalize better to unseen data.

These efficiency metrics are valuable for selecting models that balance accuracy and computational cost, ensuring that counterfeit detection systems can operate efficiently in real-world scenarios.

The four transfer learning models and our proposed autoencoder model are tested on the test dataset of 41 images including 25 authentic images and 16 counterfeit images. These models were trained twice, once using the pre-processed data and the second time using the pre-processed and augmented data. With the help of these metrics, we present that our proposed autoencoder model (AutoDetect) trained on pre-processed and augmented data greatly outperforms all of the transfer learning models.

Figure 5 shows comparison between all model combinations with and without data augmentations (DA) on F1 score, accuracy vs. computational complexity determined by the FLOPs, MACs, and trainable model parameters. It shows that our AutoDetect model with data augmentations has the highest F1 scores on both authentic and counterfeit classes while having the lowest computational complexity. It strikes the balance in this classification task by achieving an F1 score of 0.85 on authentic class and an F1 score of 0.8 on counterfeit class. The achievement becomes even more prominent considering it had the least amount of model parameters amounting to 530,000, \(\approx\) 1.34 billion FLOPs and \(\approx\) 0.67 billion MAC operations. While the EfficientNetB1 does utilize fewer FLOPs \(\approx\) 1.1 billion, it is biased towards the authentic class considering the F1 scores. Even without data augmentation, the EfficientNetB1 scores significantly lower than our model without data augmentation and is heavily biased towards the authentic class again.

Fig. 5
figure 5

Classwise F1 score and accuracy vs. model parameters, FLOPs, and MAC operations

Full size image

Similarly, Figs. 6 and 7 show the weighted average of precision recall and F1 scores versus the traininable model parameters in two different configurations: without data augmentation and with data augmentation. It shows that while our model has comparable performance to the transfer learning models in the configuration without data augmentation it still has the least amount of model parameters \(\approx\) 530,000 versus the VGG16 model which has the lowest number of model parameters \(\approx\) 6.4 million in comparison to the other transfer learning models. However, with the same number of model parameters, utilizing our model in conjuction with data augmentation significantly outperforms the other transfer learning models with a weighted average of precision recall and F1 scores of approximately 0.83. This is a 17% improvement versus the same model without data augmentation. Additionally, it is almost \(\approx\) 20% higher than the other transfer learning models utilizing data augmentation. Our model achieves this by utilizing just \(\approx\) 8% of the trainable model parameters of the lowest of the transfer learning models which is the VGG16. Figure 8 shows the scores of our best performing model configuration (AutoDetect with data augmentation) on the different classification metrics.

Fig. 6
figure 6

Model metrics vs. model parameters without data augmentation

Full size image
Fig. 7
figure 7

Model metrics vs. model parameters with data augmentation

Full size image
Fig. 8
figure 8

Classwise model metrics for all model configurations

Full size image

Figure 9 shows the amount of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) for all the model configurations. While the ResNet and EfficientNetB1 models score higher on correctly identifying the authentic class, they fail in correctly identifying the counterfeit class. Even with data augmentations, which is utilized to synthetically introduce variations in the dataset, these models fail at correctly classifying the counterfeit class. The VGG16 and InceptionV3 models strike a balance in correctly classifying both the authentic and counterfeit class but suffer from a high number of false negatives where a large number of authentic samples were incorrectly classified as counterfeit. In contrast, our model (AutoDetect) with data augmentation showed the least amount of false-positive rate that is the lowest number of counterfeit samples was incorrectly classified as authentic thus making it a suitable model for application in high security areas where the detection of counterfeit samples is of high importance. Addtionally, our model also maintains a low false-negative rate while having the highest true-negative rate in comparison to all the other models proving that our autoencoding architecture is better at detecting counterfeit samples than the traditional transfer learning models.

Fig. 9
figure 9

Confusion matrix statistics for all model configurations

Full size image

Table 3 shows the numbers of the positive and negative predictions from the different model combinations. We can see that our model AutoDetect with Data Augmentation (highlighted) gives the best balance to both authentic and counterfeit classes. This can be seen from the number of false positives and false negatives from the model. Our model was able to correctly classify 14 out 16 total counterfeits and 20 out of 25 total authentic images from the test dataset. By correctly identifying the threshold value from the reconstruction error, our model in combination with using data augmentation outperforms the other transfer learning models.

Table 3 Confusion matrix statistics
Full size table

Comparing all of these results, we demonstrate that our model in combination with data augmentation is the best performing model configuration for this task of classifying authentic and counterfeit samples from images. The reconstruction error loss used for the model along with the threshold decided to distinguish between authentic and counterfeit samples is shown in Fig. 10. This model configuration delivers the best classification accuracy in terms of correctly identifying the two classes and at the same time has the lowest number of total mis-classifications on the test data in comparison to all the other models. Also, it delivers these results by utilizing the least amount of resources thus making it the most efficient model for inference in applications where the resource overhead is to be kept at a minimum.

Fig. 10
figure 10

Reconstruction error and threshold for best performing model

Full size image

The only two mis-classifications from the best performing model in the counterfeit class were two IC images both comprising of eight pins and dual-in-line packaging as shown in Fig. 11, also, the fact that data augmentation helps to reduce the bias from the models is demonstrated in the results by reducing the cumulative error in the autoencoder models.

Fig. 11
figure 11

Counterfeits classified as authentic by AutoDetect with DA model. Both ICs have eight pins in DIP package

Full size image

6 Security Analysis

In addition to the use of pre-trained models for counterfeit IC detection [8], this paper proposes an autoencoder model to enhance security measures. Autoencoders serve as an integral component in the security framework by enabling the identification of anomalies or deviations in the data. The autoencoder model’s role in the system involves encoding input data into a compact representation and subsequently decoding it to reconstruct the original data. By training the autoencoder on genuine IC data, any deviation from this encoded representation during testing can indicate the presence of counterfeit or tampered components.

This layer of security provided by the autoencoder model acts as an alternative method to the image classification models. It helps identify potential threats and deviations in the data, making the counterfeit IC detection system more robust and resilient to adversarial attacks. By comparing the strengths of pre-trained models for classification and autoencoders for anomaly detection, the system offers a comprehensive and effective approach using autoencoder models with data augmentation for counterfeit IC detection. The data used for training, testing, and validation are sourced from the the SeCurity and AssuraNce (SCAN) Lab and are obtained from trusted facilities, ensuring that the data does not contain noisy labels. Consequently, there is no reliance on a “golden dataset” to identify counterfeit ICs, enhancing the system’s reliability. Data enhancement techniques and mapping functions play a crucial role in enhancing the model’s performance. Fine-tuning of the model’s hyperparameters for training on a smaller dataset effectively addresses the common problem of insufficient data.

Manual visual inspection, despite being a straightforward procedure to conduct, is burdened by exorbitant labor costs, ranging from $5–6 per hour in countries with low labor expenses to $30–45 per hour in regions with high labor costs. Additionally, manual inspection typically attains an 80% success rate in capturing defects, leading to potential instances of defects escaping, the quantification of which is arduous, yet they possess substantial financial consequences. Conversely, automated visual inspection presents a more exceptional performance, with recall rates that extend into the 80s and 90s and possesses the capability to identify subtle defects that may elude human operators [33]. Manual inspection of integrated circuits (ICs) is a laborious and time-consuming procedure, often taking minutes to hours per IC depending on complexity [5]. In contrast, automated inspection methods such as deep learning–based anomaly detection can analyze ICs in milliseconds to seconds [34]. This leads to significant time savings, as studies have shown a 50–90% reduction in inspection time compared to manual methods [33]. The global market for automated optical inspection is expected to grow at a CAGR of 20% [35], highlighting the industry’s shift towards faster and more efficient inspection techniques. While the initial training time and occurrence of false positives are still factors to consider, the considerable speed advantage of automated inspection makes it an appealing choice for modern IC manufacturing. AutoDetect has an inference time of 50 ms on average per IC with an F1 score in the range of 0.8 to 0.9 for both classes using the representative dataset previously mentioned. This result indicates a significant reduction in inspection time and cost as compared to manual inspection. Our future work will include conducting a survey to report the time required for manual inspection using the samples of the representative dataset and thus further validate our claim.

7 Conclusion

This paper performs a comparitive study between transfer learning approaches for counterfeit IC detection and proposes a novel autoencoder learning scheme demonstrating superlative results when used in conjuction with data augmentation versus the transfer learning counterparts. The AutoDetect model developed in this paper when used along with data augmentation outperforms the existing transfer learning models by \(\approx\) 20% while reducing the resource overhead by \(\approx\) 96% (including an average of FLOPs, MACs, and trainable parameters). We employ a variety of different classification metrics such as precision, recall, and F1 score to quantitatively and comprehensively test the accuracy of all the models. We use data augmentation as a means to rectify the imbalanced class problem in the data and to also synthetically induce variance to reduce the bias from the model predictions. This can be seen as a reduction in the cumulative error in the autoencoder model used without data augmentation and the one which used data augmentation. The transfer learning methodologies, while effective, reach a peak in the classification accuracy owing to the lack of available data and optimized training. The autoencoder learning method on the contrary can still perform with the lack of enough training data but also learn representations that the transfer learning models are incapable of learning owing to the model architecture. Thus, an autoencoder model gives the best of both worlds when it comes to increasing the classification accuracy and doing it without the availablity of a huge training dataset. As demonstrated, our autoencoder model when utilized along with the data augmentation strategies significantly outperforms the existing transfer learning models even if used in combination with data augmentation. We also believe that such models trained on representative image datasets involving authentic and counterfeits can also be translated to other IC families similar to the concept of transfer learning. A few things to consider are the fact that the model needs to be trained on a wide variety and a large number of images similar to the ImageNet dataset. The model trained should also have a high accuracy preferably in the range of 90 to 95% to ensure that when transferred to a task involving different IC families does not degrade significantly and can be tuned to perform better. The transfer learning models used in this paper are trained by a lot of compute power for multiple days to achieve extremely high accuracy and hence a model for counterfeit IC detection would also probably need enough training time and compute resources for the training process. Once that is done, it can be utilized off-the-shelf or fine-tuned for inference which, as demonstrated, can happen in the order of milliseconds.

The future research directions in this field will be guided by using novel algorithms like zero-shot learning to further bridge the gap between the lack of available training data and increasing the classification accuracy. The advancements in modeling can be used for such experiments improving the resiliency of such a counterfeit IC detection system even further.