Abstract
Barcode, including one-dimensional (1D) barcode and two-dimensional (2D) barcode, can be seen almost anywhere in our lives. In many barcode-based mobile systems, different barcodes will appear simultaneously with different angles, shapes, and image quality. Barcode localization is a significant prerequisite for barcode decoding in these applications. In this paper, we propose a region-based end-to-end network to finely localize and classify 1D barcode and Quick Response (QR) code in complex environments. Two special layers are designed in our network. One is a quadrilateral regression layer to localize arbitrary quadrilateral bounding boxes, and another is a Multi-scale Spatial Pyramid Pooling (MSPP) layer to improve the detection accuracy of small-scale barcodes. Extensive experiments on existing public datasets and our own dataset have verified the effectiveness of proposed layers. We also demonstrate that our method can resist some distortions by simulating barcode images of different image qualities. What’s more, a human decoding experiment is also performed to prove the effectiveness of our method as a preprocessor for QR code decoding.
Similar content being viewed by others
References
Al-Khalifa HS (2008) Utilizing qr code and mobile phones for blinds and visually impaired people. In: International conference on computers helping people with special needs (ICCHP), pp 1065–1069
Anezaki T, Eimon K, Tansuriyavong S, Yagi Y (2011) Development of a human-tracking robot using qr code recognition. In: Frontiers of computer vision (FCV), pp 1–6
Ardón P, Dragone M, Erden MS (2018) Reaching and grasping of objects by humanoid robots through visual servoing. In: Haptic: science, technology, and applications, pp 353–365
Belussi L, Hirata N (2011) Fast qr code detection in arbitrarily acquired images. In: SIBGRAPI conference on graphics, patterns and images (SIBGRAPI), pp 281–288
Busta M, Neumann L, Matas J (2017) Deep textspotter: an end-to-end trainable scene text localization and recognition framework. In: The IEEE international conference on computer vision (ICCV)
Cai Z, Fan Q, Feris RS, Vasconcelos N (2016) A unified multi-scale deep convolutional neural network for fast object detection. In: European conference on computer vision (ECCV), pp 354–370
Chang Y, Tsai S, Chang Y, Wang T (2007) A novel wayfinding system based on geo-coded qr codes for individuals with cognitive impairments. In: International ACM SIGACCESS conference on computers and accessibility, pp 231–232
Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. In: Advances in neural information processing systems (NIPS), pp 379–387
Dollár P, Appel R, Belongie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell (PAMI) 36(8):1532–1545
Dubská M, Herout A, Havel J (2016) Real-time precise detection of regular grids and matrix codes. J Real-Time Image Process (JRTIP), 193–200
Ebrahim Y, Abdelsalam W, Ahmed M, Chau SC (2005) Proposing a hybrid tag-camera-based identification and navigation aid for the visually impaired. In: Second IEEE consumer communications and networking conference (CCNC), pp 172–177
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on computer vision and pattern recognition, pp 580–587. https://doi.org/10.1109/CVPR.2014.81
Grósz T, Bodnár P, Tóth L, Nyúl LG (2014) Qr code localization using deep neural networks. In: IEEE International workshop on machine learning for signal processing (MLSP), pp 1–6
Hansen DK, Nasrollahi K, Rasmusen CB, Moeslund TB (2017) Real-time barcode detection and classification using deep learning. In: International joint conference on computational intelligence (IJCCI), pp 321–327
He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell (PAMI) 37(9):1904–1916
Hu GX, Yang Z, Hu L, Huang L, Han JM (2018) Small object detection with multiscale features. International Journal of Digital Multimedia Broadcasting (JDMB
Katona M, Nyúl LG (2012) A novel method for accurate and efficient barcode detection with morphological operations. In: International conference on signal image technology and internet based systems (SITIS), pp 307–314
Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: The IEEE conference on computer vision and pattern recognition (CVPR)
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision (ECCV), pp 21–37
Ma J, Shao W, Ye H, Wang L, Wang H, Zheng Y, Xue X (2018) Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans Multimed (TMM) 20(11):3111–3122
Nguyen TV, Le T, Bui Q, Tran M, Duong A (2012) Smart shopping assistant: a multimedia and social media augmented system with mobile devices to enhance customers’ experience and interaction. In: Pacific Asia conference on information systems (PACIS) 2012 proceedings, p 95
Pandya KH, Galiyawala HJ (2014) A survey on qr codes: in context of research and application. Int J Emerg Technol Adv Eng (JETAE) 4(3):258–262
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified real-time object detection. In: The IEEE Conference on computer vision and pattern recognition (CVPR)
Ren S, He R, Girshick K, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Neural information processing systems (NIPS), pp 91–99
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:abs/1409.1556
Sörös G, Flörkemeier C (2013) Blur-resistant joint 1d and 2d barcode localization for smartphones. In: International conference on mobile and ubiquitous multimedia (MUM), pp 11:1–11:8
Szentandrási I, Herout A, Dubská M (2012) Fast detection and recognition of qr codes in high-resolution images. In: Conference on computer graphics (CCG), pp 129–136
Uijlings JRR, van de Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171. https://doi.org/10.1007/s11263-013-0620-5
Wachenfeld S, Terlunen S, Jiang X (2008) Robust recognition of 1-d barcodes using camera phones. In: International conference on pattern recognition (ICPR), pp 1–4
Walsh A (2010) Qr codes – using mobile phones to deliver library instruction and help at the point of need. J Inform Liter 4:1
Xu W, McCloskey S (2011) 2d barcode localization and motion deblurring using a flutter shutter camera. In: IEEE Workshop on applications of computer vision (WACV), pp 159–165
Zamberletti A, Gallo I, Albertini S (2013) Robust angle invariant 1d barcode detection. In: Asian conference on pattern recognition (ACPR), pp 160–164
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, J., Min, X., Jia, J. et al. Fine localization and distortion resistant detection of multi-class barcode in complex environments. Multimed Tools Appl 80, 16153–16172 (2021). https://doi.org/10.1007/s11042-019-08578-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-08578-x