ViT-SigNet: Combining Deep CNN and Vision Transformer for Enhanced Signature Verification

Abstract

In the digital era, digital signatures are crucial for online processes and transactions. Their blend of security and ease has spurred various verification methods, including machine and deep learning models. Vision Transformers (ViT) have recently gained traction for image tasks due to their effectiveness. This study presents a signature verification method combining a custom ResNet-50 Convolutional Neural Network (CNN) and Transformers. This combined model leverages ResNet-50’s image extraction capabilities and the Transformer’s attention mechanism. We optimized the model with learning rate adjustments, enhancing its performance. Our data included 534 personal signatures from students, split into 128 for training and 406 for testing. We used machine learning for feature extraction and classification, assessing genuineity with cosine similarity. We tested various CNNs, ViT, and our combined model. Paired with XGBoost, our model achieved a 97.7% precision, 98.45% recall, and 97.5% F1 score. It also performed well in signature verification metrics like FNMR 0.090, FMR 0.059, and EER 0.075. Our method shows promise for advancing signature verification, suggesting potential future research directions.