Abstract
Object detection is a popular computer vision task that is performed by autonomous industrial robots. However, training a detection model requires a large annotated image dataset that belongs to the camera domain of the robot (the test domain). Acquiring such data in a similar domain or rendering photo-realistic images from a realistic virtual environment composed of accurate 3D models and using powerful hardware, can be expensive, time-consuming, and requires specialized expertise. This article focuses on investigating the growth of average precision (AP) in object detection as we progressively train and test our models using various combinations of acquired and rendered datasets from different domains: real and synthetic. By analyzing the results on industrial load carrier box detection, we discovered that a hybrid dataset comprising 20–30% of images similar to the test domain leads to achieving nearly maximum detection accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The real image dataset cannot be shared due to confidential reasons. Only, the rendered synthetic dataset is available upon a valid request from the corresponding author.
- 2.
R has clearly shown better results on R only (\(AP_{R+S:R}\) = 90.74% at r = 1.00) compared to its performance on S (\(AP_{S+R:S}\) = 5.49% at r = 0.00).
References
Abou Akar, C., Tekli, J., Jess, D., Khoury, M., Kamradt, M., Guthe, M.: Synthetic object recognition dataset for industries. In: 2022 35th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), vol. 1, pp. 150–155. IEEE (2022)
Antoniou, A., Storkey, A., Edwards, H.: Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04340 (2017)
Gaidon, A., Wang, Q., Cabon, Y., Vig, E.: Virtual worlds as proxy for multi-object tracking analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4340–4349 (2016)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Greff, K., et al.: Kubric: a scalable dataset generator. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3749–3761 (2022)
Hinterstoisser, S., Pauly, O., Heibel, H., Martina, M., Bokeloh, M.: An annotation saved is an annotation earned: Using fully synthetic training for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019)
Johnson-Roberson, M., Barto, C., Mehta, R., Sridhar, S.N., Rosaen, K., Vasudevan, R.: Driving in the matrix: can virtual worlds replace human-generated annotations for real world tasks?. arXiv preprint arXiv:1610.01983 (2016)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Meng, J., et al.: The future of computer vision. APSIPA Trans. Signal Inf. Process. 11(1) (2022)
Morrical, N., et al.: Nvisii: A scriptable tool for photorealistic image generation. arXiv preprint arXiv:2105.13962 (2021)
Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 (2017)
Petrovic, O., Duarte, D.L.D., Storms, S., Herfs, W.: Towards knowledge-based generation of synthetic data by taxonomizing expert knowledge in production. In: Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems, vol. 69, no. 69 (2023)
PNVR, K., Zhou, H., Jacobs, D.: Sharingan: combining synthetic and real data for unsupervised geometry estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13974–13983 (2020)
Ravuri, S., Vinyals, O.: Classification accuracy score for conditional generative models. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: ground truth from computer games. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 102–118. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_7
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3234–3243 (2016)
Seib, V., Lange, B., Wirtz, S.: Mixing real and synthetic data to enhance neural network training–a review of current approaches. arXiv preprint arXiv:2007.08781 (2020)
Saleh, F.S., Aliakbarian, M.S., Salzmann, M., Petersson, L., Alvarez, J.M.: Effective use of synthetic data for urban scene semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 84–100 (2018)
Sankaranarayanan, S., Balaji, Y., Jain, A., Lim, S.N., Chellappa, R.: Learning from synthetic data: addressing domain shift for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3752–3761 (2018)
Seib, V., Lange, B., Wirtz, S.: Mixing real and synthetic data to enhance neural network training-a review of current approaches. arXiv preprint arXiv:2007.08781 (2020)
Shao, S., et al.: Objects365: a large-scale, high-quality dataset for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8430–8439 (2019)
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., Abbeel, P.: Domain randomization for transferring deep neural networks from simulation to the real world. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23–30. IEEE (2017)
Vu, T.H., Jain, H., Bucher, M., Cord, M., Pérez, P.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2517–2526 (2019)
Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: Posecnn: a convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199 (2017)
Yao, Y., Zheng, L., Yang, X., Naphade, M., Gedeon, T.: Simulating content consistent vehicle datasets with attribute descent. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12351, pp. 775–791. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58539-6_46
Zabir, M., Fazira, N., Ibrahim, Z., Sabri, N.: Evaluation of pre-trained convolutional neural network models for object recognition. Int. J. Eng. Technol. 7(3.15), 95–98 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Abou Akar, C., Semaan, A., Haddad, Y., Kamradt, M., Makhoul, A. (2023). Mixing Domains for Smartly Picking and Using Limited Datasets in Industrial Object Detection. In: Christensen, H.I., Corke, P., Detry, R., Weibel, JB., Vincze, M. (eds) Computer Vision Systems. ICVS 2023. Lecture Notes in Computer Science, vol 14253. Springer, Cham. https://doi.org/10.1007/978-3-031-44137-0_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-44137-0_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44136-3
Online ISBN: 978-3-031-44137-0
eBook Packages: Computer ScienceComputer Science (R0)