Abstract
Purpose
Automated assessment of dairy cow traits, important for productivity evaluation, provides advantages by mitigating personal biases, measurement errors, and stress factors typically associated with manual assessment. To develop such a system, the initial step involves accurately segmenting cow body regions for subsequent trait measurement.
Methods
Thus, the present study introduces a refined DeepLabV3 + CNN model with EfficientNetB2 as the backbone and enhanced with attention mechanisms, aiming for precise segmentation of cow body regions from lateral and posterior views. In the DeepLabV3 + model, various backbone models, including MobileNet, MobileNetV2, MobileNetV3, EfficientNetB0, EfficientNetB1, and EfficientNetB2, were evaluated. Among these, EfficientNetB2 exhibited superior performance in lateral view segmentation, achieving a mean Intersection-over-union (m-IoU) of 94.19%. To further enhance segmentation accuracy, attention mechanisms such as Squeeze and Excitation (SE), Residual connection-infused Squeeze and Excitation (SER), Convolutional Block Attention Module (CBAM), and Residual connection-infused Convolutional Block Attention Module (CBAMR) were incorporated into the DeepLabV3 + model.
Results
The introduction of attention mechanisms in the EfficientNetB2 model led to enhanced m-IoU values: SE (94.27%), SER (94.25%), CBAM (94.59%), and CBAMR (94.66%). EfficientNetB2, integrated with CBAM and Residual connections (termed CBAMR), found to be top-performing model, achieving m-IoU values of 94.66% (lateral view), 93.77% (posterior view), and 99.61% (stature). The lateral view segmentation demonstrated high IoU for the body (98.73%) and rump (96.54%), with lowest IoU for teats (79.70%) due to their smaller spatial presence in input image. For posterior view regions, the CBAMR model achieved IoU scores above 79.0%, with the rear leg showing the highest (96.70%) and rump bones the lowest (79.52%). The segmentation accuracy for stature exceeded 90.0%, indicating less complexity in single-body region segmentation.
Conclusions
Therefore, these developed models demonstrate considerable accuracy in segmenting cow regions, making a significant contribution to the advancement of computer vision–based systems for measuring linear-type traits, and hold promise for deployment in such an automatic system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The data that support the findings of this study are available from the corresponding author, upon reasonable request.
References
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kudlur, M., Levenberg, J., Monga, R., Moore, S., Murray, D. G., Steiner, B., Tucker, P., Vasudevan, V., Warden, P., Wicke, M., Yu, Y., & Zheng, X. (2016). TensorFlow: A system for large-scale machine learning. Business Opp, 10, 265–283. https://doi.org/10.48550/arXiv.1605.08695
Azizi, A., Abbaspour-Gilandeh, Y., Vannier, E., Dusséaux, R., Mseri-Gundoshmian, T., & Moghaddam, H. A. (2020). Semantic segmentation: A modern approach for identifying soil clods in precision farming. Biosystems Engineering, 196, 172–182. https://doi.org/10.1016/j.biosystemseng.2020.05.022
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615
Batanov, S. D., Starostina, O. S., & Baranova, I. A. (2019). Non-contact methods of cattle conformation assessment using mobile measuring systems. IOP Conference Series: Earth and Environmental Science, 315, 032006. https://doi.org/10.1088/1755-1315/315/3/032006
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 834–848. https://doi.org/10.1109/TPAMI.2017.2699184
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected CRFs. https://doi.org/10.48550/ARXIV.1412.7062
Chen, L.-C., Papandreou, G., Schroff, F., Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. https://doi.org/10.48550/ARXIV.1706.05587
Chollet, F., (2015). Keras. https://github.com/keras-team/keras. Accessed 10/03/2024
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. Presented at the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), IEEE, Miami, FL, pp. 248–255. https://doi.org/10.1109/CVPR.2009.5206848
Du, A., Guo, H., Lu, J., Su, Y., Ma, Q., Ruchay, A., Marinello, F., & Pezzuolo, A. (2022). Automatic livestock body measurement based on keypoint detection with multiple depth cameras. Computers and Electronics in Agriculture, 198, 107059. https://doi.org/10.1016/j.compag.2022.107059
Feng, T., Guo, Y., Huang, X., & Qiao, Y. (2023). Cattle target segmentation method in multi-scenes using improved DeepLabV3+ method. Animals, 13, 2521. https://doi.org/10.3390/ani13152521
Fernandes, A. F. A., Turra, E. M., De Alvarenga, É. R., Passafaro, T. L., Lopes, F. B., Alves, G. F. O., Singh, V., & Rosa, G. J. M. (2020). Deep learning image segmentation for extraction of fish body measurements and prediction of body weight and carcass traits in Nile tilapia. Computers and Electronics in Agriculture, 170, 105274. https://doi.org/10.1016/j.compag.2020.105274
Fujii, H., Tanaka, H., Ikeuchi, M., Hotta, K. (2021). X-net with different loss functions for cell image segmentation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Presented at the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 3788–3795. https://doi.org/10.1109/CVPRW53098.2021.00420
Gonzalez-Huitron, V., León-Borges, J. A., Rodriguez-Mata, A. E., Amabilis-Sosa, L. E., Ramírez-Pereda, B., & Rodriguez, H. (2021). Disease detection in tomato leaves via CNN with lightweight architectures implemented in Raspberry Pi 4. Computers and Electronics in Agriculture, 181, 105951. https://doi.org/10.1016/j.compag.2020.105951
Google Colaboratory. (2021). Google Colaboratory [WWW Document]. URL https://colab.research.google.com/notebooks/basic_features_overview.ipynb (accessed 11.14.21).
Guvenoglu, E. (2023). Determination of the live weight of farm animals with deep learning and semantic segmentation techniques. Applied Sciences, 13, 6944. https://doi.org/10.3390/app13126944
Haggag, H., Abobakr, A., Hossny, M., Nahavandi, S. (2016). Semantic body parts segmentation for quadrupedal animals. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Presented at the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, Budapest, Hungary, pp. 000855–000860. https://doi.org/10.1109/SMC.2016.7844347
Hanh, B. T., Van Manh, H., & Nguyen, N.-V. (2022). Enhancing the performance of transferred efficientnet models in leaf image-based plant disease classification. Journal of Plant Diseases and Protection, 129, 623–634. https://doi.org/10.1007/s41348-022-00601-y
Harahap, S. A. F., & Irmawan, I. (2024). Performance comparison of MobileNet, EfficientNet, and Inception for predicting crop disease. Selco, 1, 30–36. https://doi.org/10.62420/selco.v1i1.4
He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Las Vegas, NV, USA, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. https://doi.org/10.48550/ARXIV.1704.04861
Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, Q.V., Adam, H. (2019). Searching for MobileNetV3. https://doi.org/10.48550/ARXIV.1905.02244
Jia, N., Kootstra, G., Koerkamp, P. G., Shi, Z., & Du, S. (2021). Segmentation of body parts of cows in RGB-depth images based on template matching. Computers and Electronics in Agriculture, 180, 105897. https://doi.org/10.1016/j.compag.2020.105897
Kingma, D. P., & Ba, J. (2017). Adam: A method for stochastic optimization. https://doi.org/10.48550/arXiv.1412.6980
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097–1105. https://doi.org/10.1145/3065386
Kumar, A., Jain, R., & Dwivedi, R. (2023). Weed detection in crops using lightweight EfficientNets. lecture notes in networks and systemsIn H. Sharma, V. Shrivastava, K. K. Bharti, & L. Wang (Eds.), Communication and intelligent systems (pp. 149–162). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-99-2100-3_13
Liao, F., Feng, X., Li, Z., Wang, D., Xu, C., Chu, G., Ma, H., Yao, Q., Chen, S. (2023). A hybrid CNN-LSTM model for diagnosing rice nutrient levels at the rice panicle initiation stage. Journal of Integrative Agriculture. https://doi.org/10.1016/j.jia.2023.05.032
Liu, S., Li, M., Li, M., & Xu, Q. (2020). Research of animals image semantic segmentation based on deep learning. Concurrency and Computation, 32, e4892. https://doi.org/10.1002/cpe.4892
Martins, G. B., La Rosa, L. E. C., Happ, P. N., Filho, L. C. T. C., Santos, C. J. F., Feitosa, R. Q., & Ferreira, M. P. (2021). Deep learning-based tree species mapping in a highly diverse tropical urban setting. Urban Forestry & Urban Greening, 64, 127241. https://doi.org/10.1016/j.ufug.2021.127241
Naik, B. N., Malmathanraj, R., & Palanisamy, P. (2022). Detection and classification of chilli leaf disease using a squeeze-and-excitation-based CNN model. Ecological Informatics, 69, 101663. https://doi.org/10.1016/j.ecoinf.2022.101663
Nanni, L., Ghidoni, S., & Brahnam, S. (2017). Handcrafted vs. non-handcrafted features for computer vision classification. Pattern Recognition, 71, 158–172. https://doi.org/10.1016/j.patcog.2017.05.025
Ni, X., Takeda, F., Jiang, H., Yang, W. Q., Saito, S., & Li, C. (2022). A deep learning-based web application for segmentation and quantification of blueberry internal bruising. Computers and Electronics in Agriculture, 201, 107200. https://doi.org/10.1016/j.compag.2022.107200
Nye, J., Zingaretti, L. M., & Pérez-Enciso, M. (2020). Estimating conformational traits in dairy cattle with DeepAPS: A two-step deep learning automated phenotyping and segmentation approach. Frontiers in Genetics, 11, 513. https://doi.org/10.3389/fgene.2020.00513
Qian, D., Wang, W., Huo, X., & Tang, J. (2008). Study on linear appraisal of dairy cow’s conformation based on image processing. The International Federation for Information ProcessingIn D. Li (Ed.), Computer and computing technologies in agriculture (Vol. I, pp. 303–311). US, Boston, MA: Springer. https://doi.org/10.1007/978-0-387-77251-6_33
Qiao, Y., Clark, C., Lomax, S., Kong, H., Su, D., & Sukkarieh, S. (2021). Automated individual cattle identification using video data: A unified deep learning architecture approach. Frontiers in Animal Science, 2, 759147. https://doi.org/10.3389/fanim.2021.759147
Qiao, Y., Su, D., Kong, H., Sukkarieh, S., Lomax, S., Clark, C. (2020). Data augmentation for deep learning based cattle segmentation in precision livestock farming. In 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE). Presented at the 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE), IEEE, Hong Kong, Hong Kong, pp. 979–984. https://doi.org/10.1109/CASE48305.2020.9216758
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. https://doi.org/10.48550/arXiv.1505.04597
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. https://doi.org/10.48550/ARXIV.1801.04381
Santos, T. T., De Souza, L. L., Dos Santos, A. A., & Avila, S. (2020). Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association. Computers and Electronics in Agriculture, 170, 105247. https://doi.org/10.1016/j.compag.2020.105247
Schneider, M. P., Dürr, J. W., Cue, R. I., & Monardes, H. G. (2003). Impact of type traits on functional herd life of Quebec Holsteins assessed by survival analysis. Journal of Dairy Science, 86, 4083–4089. https://doi.org/10.3168/jds.S0022-0302(03)74021-1
Shah, D., Trivedi, V., Sheth, V., Shah, A., & Chauhan, U. (2022). ResTS: Residual deep interpretable architecture for plant disease detection. Information Processing in Agriculture, 9, 212–223. https://doi.org/10.1016/j.inpa.2021.06.001
Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. https://doi.org/10.48550/arXiv.1409.1556
Singh, N., Tewari, V. K., Biswas, P. K., Pareek, C. M., & Dhruw, L. K. (2021). Image processing algorithms for in-field cotton boll detection in natural lighting conditions. Artificial Intelligence in Agriculture, 5, 142–156. https://doi.org/10.1016/j.aiia.2021.07.002
Singh, N., Tewari, V. K., Biswas, P. K., Dhruw, L. K., Pareek, C. M., & Singh, H. D. (2022). Semantic segmentation of in-field cotton bolls from the sky using deep convolutional neural networks. Smart Agricultural Technology, 2, 100045. https://doi.org/10.1016/j.atech.2022.100045
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2015). Rethinking the Inception architecture for computer vision. https://doi.org/10.48550/arXiv.1512.00567
Tan, M., Le, Q.V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. https://doi.org/10.48550/ARXIV.1905.11946
Tang, J., Zhao, Y., Feng, L., & Zhao, W. (2022). Contour-based wild animal instance segmentation using a few-shot detector. Animals, 12, 1980. https://doi.org/10.3390/ani12151980
Tasdemir, S., Urkmez, A., & Inal, S. (2011). Determination of body measurements on the Holstein cows using digital image analysis and estimation of live weight with regression analysis. Computers and Electronics in Agriculture, 76, 189–197. https://doi.org/10.1016/j.compag.2011.02.001
Tkachenko, M., Malyuk, M., Holmanyuk, A., & Liubimo, N. (2020). Label Studio: Data labeling software. https://labelstud.io/. Accessed 08/03/2024
Tsalera, E., Papadakis, A., Samarakou, M., & Voyiatzis, I. (2022). Feature extraction with handcrafted methods and convolutional neural networks for facial emotion recognition. Applied Sciences, 12, 8455. https://doi.org/10.3390/app12178455
Weales, D., Moussa, M., & Tarry, C. (2021). A robust machine vision system for body measurements of beef calves. Smart Agricultural Technology, 1, 100024. https://doi.org/10.1016/j.atech.2021.100024
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018). CBAM: Convolutional Block Attention Module. Lecture Notes in Computer ScienceIn V. Ferrari, M. Hebert, C. Sminchisescu, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 (pp. 3–19). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-01234-2_1
Wu, D., Yin, X., Jiang, B., Jiang, M., Li, Z., & Song, H. (2020). Detection of the respiratory rate of standing cows by combining the Deeplab V3+ semantic segmentation model with the phase-based video magnification algorithm. Biosystems Engineering, 192, 72–89. https://doi.org/10.1016/j.biosystemseng.2020.01.012
Zhang, A. L., Pei Wu, B., Tana Wuyun, C., Xinhua Jiang, D., Chuanzhong Xuan, E., & Yanhua Ma, F. (2018). Algorithm of sheep body dimension measurement and its applications based on image analysis. Computers and Electronics in Agriculture, 153, 33–45. https://doi.org/10.1016/j.compag.2018.07.033
Zhang, J., Zhuang, Y., Ji, H., & Teng, G. (2021). Pig weight and body size estimation using a multiple output regression convolutional neural network: A fast and fully automatic method. Sensors, 21, 3218. https://doi.org/10.3390/s21093218
Acknowledgements
We are thankful for the support and facilities extended by the Director of ICAR-NDRI and the workforce involved in data collection, which contributed towards the successful completion of this research. Our special thanks are also due to the Director of ICAR—Research Complex for North Eastern Hilly Region, Umiam, Meghalaya, India, for providing essential facilities for conducting the study.
Funding
This study received financial grant from the Science Engineering Research Board (SERB), New Delhi, India (SRG/2020/001804).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Singh, N., Devi, I., Dudi, K. et al. Development of Attention-Enabled Multi-Scale Pyramid Network-Based Models for Body Part Segmentation of Dairy Cows. J. Biosyst. Eng. (2024). https://doi.org/10.1007/s42853-024-00226-z
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42853-024-00226-z