Handwritten Urdu Recognition Using BERT with Vision Transformers

Abstract

The handwritten Urdu text recognition is a challenging area in pattern recognition and has gained much importance after the rapid emergence of several camera-based applications on portable devices, which facilitate the daily processing of plenty of images. The various challenges encountered in handwritten Urdu recognition are writer-dependent variations amongst different Urdu writers, irregular positioning of diacritics associated with a character, context sensitivity of characters, and cursive nature of Urdu script, that also enhances the difficulty in the formulation of a large handwritten Urdu dataset. Several researchers have proposed handwritten Urdu databases but none of them can individually cover the entire Urdu ligature corpus. Moreover, the handwritten Urdu recognition rate obtained by these researchers has been determined to be poor once their approaches are applied on different Urdu datasets. Hence a novel Transformer based methodology using BERT architecture has been proposed in this research. The proposed approach uses convolution feature maps as word embedding in the transformer that makes full use of powerful attention mechanism of the transformer to focus on handwritten data. To cover the entire Urdu corpus, we have evaluated all available handwritten Urdu datasets for experimental evaluation to determine the various performance parameters. The methodology is computationally efficient with a small constant time complexity and reveal a better recognition rate of 98% on similar datasets and 95% on dissimilar datasets.