Abstract
Detection of respiratory symptoms has long been an area of extensive research to expedite the process of machine aided diagnosis for various respiratory conditions. This chapter attempts to address the early diagnosis of respiratory conditions using low power scalable software and hardware involving end-to-end convolutional neural networks (CNNs). We propose RespiratorNet, a scalable multimodal CNN software hardware architecture that can take audio recordings, speech information, and other sensor modalities belonging to patient demographic or symptom information as input to classify different respiratory symptoms. We analyze four different publicly available datasets and use them as case studies as part of our experiment to classify respiratory symptoms. With regards to fitting the network architecture to the hardware framework, we perform windowing, low bit-width quantization, and hyperparameter optimization on the software side. As per our analysis, detection accuracy goes up by 5% when patient demographic information is included in the network architecture. The hardware prototype is designed using Verilog HDL on Xilinx Artix-7 100t FPGA with hardware scalability extending to accommodate different numbers of processing engines for parallel processing. The proposed hardware implementation has a low power consumption of only 245 mW and achieves an energy efficiency of 7.3 GOPS/W which is 4.3 better than the state-of the-art accelerator implementations. In addition, RespiratorNet TensorFlow model is implemented on NVIDIA Jetson TX2 SoC (CPU+GPU) and compared to TX2 single-core CPU and GPU implementations to provide scalability in terms of off-the-shelf platform implementations.
Keywords
- Multimodal CNN
- Scalable respiratory symptoms detection
- Low power embedded
- Audio detection
- FPGA
This is a preview of subscription content, access via your institution.
Buying options
References
Zhao, X., Zhang, B., Li, P., Ma, C., Gu, J., Hou, P., Guo, Z., Wu, H., Bai, Y.: Incidence, clinical characteristics and prognostic factor of patients with COVID-19: a systematic review and meta-analysis. MedRxiv (2020)
Lee, P.-I., Hu, Y.-L., Chen, P.-Y., Huang, Y.-C., Hsueh, P.-R.: Are children less susceptible to COVID-19? J. Microbiol. Immunol. Infect. (2020)
Cho, S.-H., Lin, H.-C., Ghoshal, A.G., Muttalif, A.R.B.A., Thanaviratananich, S., Bagga, S., Faruqi, R., Sajjan, S., Brnabic, A.J.M., Dehle, F.C., et al.: Respiratory disease in the Asia-Pacific region: cough as a key symptom. In: Allergy & Asthma Proceedings, vol. 37
Korpáš, J., Tomori, Z.: Cough and Other Respiratory Reflexes./Kasˇel’ a Ine´ Respiracˇne´ Reflexy. Veda (1979)
Korpáš, J., Sadlonˇova´, J., Vrabec, M.: Analysis of the cough sound: an overview. Pulmonary Pharmacol. 9(5–6), 261–268 (1996)
Amoh, J., Odame, K.: DeepCough: a deep convolutional neural net-work in a wearable cough detection system. In: 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS). IEEE, pp. 1–4 (2015)
Ren, H., et al.: End-to-end scalable and low power multi-modal CNN for respiratory-related symptoms detection. In: 2020 IEEE 33rd International System- on-Chip Conference (SOCC) (SOCC 2020)
Mazumder, A.N., Ren, H., Rashid, H.-A., Hosseini, M., Chandrareddy, V., Homayoun, H., Mohsenin, T.: Automatic detection of respiratory symptoms using a low power multi-input CNN processor. IEEE Des. Test 2021, 1–1 (2021). https://doi.org/10.1109/MDAT.20213079318
Hosseini, M., Ren, H., Rashid, H., Mazumder, A., Prakash, B., Mohsenin, T.: Neural networks for pulmonary disease diagnosis using auditory and demographic information. In: epiDAMIK 2020: 3rd epiDAMIK ACM SIGKDD International Workshop on Epidemiology meets Data Mining and Knowledge Discovery. ACM, pp. 1–5, in press
Jafari, A., et al.: SensorNet: a scalable and low-power deep convolutional neural network for multimodal data classification. IEEE Trans. Circ. Syst. I Reg. Papers 66(1), 274–287 (2019). https://doi.org/10.1109/TCSI.2018.2848647
Rashid, H.-A., Manjunath, N.K., Paneliya, H., Hosseini, M., Mohsenin, T.: A low-power LSTM processor for multi-channel brain EEG artifact detection. In: 2020 21th International Symposium on Quality Electronic Design (ISQED). IEEE (2020)
Shea, C., Page, A., Mohsenin, T.: SCALENet: a scalable low power accelerator for real-time embedded deep neural networks. In: ACM Proceedings of the 28th Edition of the Great Lakes Symposium on VLSI (GLSVLSI). ACM (2018)
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., Yu, D.: Convolutional neural networks for speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 22(10), 1533–1545 (2014)
Piczak, K.J.: Environmental sound classification with convolutional neural networks. In: 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, pp. 1–6 (2015)
Amoh, J., Odame, K.: Deep neural networks for identifying cough sounds. IEEE Trans. Biomed. Circ. Syst. 10(5), 1003–1011 (2016)
Nakano, H., Furukawa, T., Tanigawa, T.: Tracheal sound analysis using a deep neural network to detect sleep apnea. J. Clin. Sleep Med. 15(8), 1125–1133 (2019)
Ryu, H., Park, J., Shin, H.: Classification of heart sound recordings using convolution neural network. In: 2016 Computing in Cardiology Conference (CinC). IEEE, pp. 1153–1156 (2016)
Aykanat, M., Kurt, O.K.B., Saryal, S.: Classification of lung sounds using convolutional neural networks. EURASIP J. Image Video Process. 1, 65 (2017)
Perna, D., Tagarelli, A.: Deep auscultation: predicting respiratory anomalies and diseases via recurrent neural networks. In: 2019 IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS). IEEE, pp. 50–55 (2019)
Rocha, B.M., Filos, D., Mendes, L., Vogiatzis, I., Perantoni, E., Kaimakamis, E., Natsiavas, P., Oliveira, A., Ja´come, C., Marques, A., et al.: A respiratory sound database for the development of automated classification. In: International Conference on Biomedical and Health Informatics. Springer, pp. 33–37 (2017)
Liu, R., Cai, S., Zhang, K., Hu, N.: Detection of adventitious respiratory sounds based on convolutional neural network. In: 2019 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS). IEEE, pp. 298–303 (2019)
Perna, D.: Convolutional neural networks learning from respiratory data. In: 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, pp. 2109–2113 (2018)
Pham, L., McLoughlin, I., Phan, H., Tran, M., Nguyen, T., Palaniappan, R.: Robust Deep Learning Framework For Predicting Respiratory Anomalies and Diseases (2020). arXiv:2002.03894
Demir, F., Sengur, A., Bajaj, V.: Convolutional neural networks based efficient approach for classification of lung diseases. Health Inf. Sci. Syst. 8(1), 4 (2020)
Acharya, J., Basu, A.: Deep neural network for respiratory sound classification in wearable devices enabled by patient specific model tuning. IEEE Trans. Biomed. Circ. Syst. 14(3), 535–544 (2020)
Laguarta, J., Hueto, F., Subirana, B.: COVID-19 artificial intelligence diagnosis using only cough recordings. IEEE Open J. Eng. Med. Biol. (2020)
Fonseca, E., Plakal, M., Font, F., Ellis, D.P.W., Favory, X., Pons, J., Serra, X.: General-purpose tagging of freesound au- dio with audioset labels: Task description, dataset, and baseline (2018). arXiv:1807.09902
Piczak, K.J.: ESC: Dataset for environmental sound classification. In: Proceedings of the 23rd ACM international conference on Multimedia, pp. 1015–1018 (2015)
Orlandic, L., Teijeiro, T., Atienza, D.: The COUGHVID crowd- sourcing dataset: A corpus for the study of large-scale cough analysis algorithms (2020). arXiv:2009.11644
Abadi, M. et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems (2015). http://www.tensorflow.org/
McFee, B., Raffel, C., Liang, D., Ellis, D.P.W., Matt McVicar, Battenberg, E., Nieto, O.: librosa: Audio and Music Signal Analysis in Python (2015)
Rocha, B.M., Filos, D., Mendes, L., Serbes, G., Ulukaya, S., Kahya, Y.P., Jakovljevic, N., Turukalo, T.L., Vogiatzis, I.M., Perantoni, E., et al.: An open access database for the evaluation of respiratory sound classification algorithms. Physiol. Measur. 40(3), 035001 (2019)
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., Cong, J.: Optimizing FPGA-based accelerator design for deep convolutional neural net- works. In: Proceedings of the 2015 ACM/SIGDA international symposium on field- programmable gate arrays, pp. 161–170 (2015)
Ma, Y., Suda, N., Cao, Y., Seo, J., Vrudhula, S.: Scalable and modularized RTL compilation of convolutional neural networks onto FPGA. In : 2016 26th International Conference on Field Programmable Logic and Applications (FPL). IEEE, pp. 1–8 (2016)
Huang, C., Ni, S., Chen, G.: A layer-based structured design of CNN on FPGA. In: 2017 IEEE 12th International Conference on ASIC (ASICON). IEEE, 1037–1040 (2017)
Acknowledgements
We acknowledge the support of the University of Maryland, Baltimore, Institute for Clinical Translational Research (ICTR) and the National Center for Advancing Translational Sciences (NCATS) Clinical Translational Science Award (CTSA) grant number UL1TR003098.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Rashid, HA., Ren, H., Mazumder, A.N., Sajadi, M.M., Mohsenin, T. (2022). A Re-configurable Software-Hardware CNN Framework for Automatic Detection of Respiratory Symptoms. In: Adibi, S., Rajabifard, A., Shariful Islam, S.M., Ahmadvand, A. (eds) The Science behind the COVID Pandemic and Healthcare Technology Solutions. Springer Series on Bio- and Neurosystems, vol 15. Springer, Cham. https://doi.org/10.1007/978-3-031-10031-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-10031-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10030-7
Online ISBN: 978-3-031-10031-4
eBook Packages: MedicineMedicine (R0)