Skip to main content

Advertisement

SpringerLink
Log in

Deep transfer learning algorithms applied to synthetic drawing images as a tool for supporting Alzheimer’s disease prediction

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Neurodegenerative diseases, such as Alzheimer’s Disease or Parkinson’s disease, are unfortunately still incurable, although there are many therapies that can slow down the progression of the disease and improve patients’ lives. An essential condition, however, is the early diagnosis of these disorders to begin therapies as soon as possible: In fact, when the signs of the disease become evident, damages may be already significant and irreversible. In this context, it is generally agreed that handwriting is one of the first skills affected by the onset of cognitive disorders. For this reason, in a preliminary study, we considered a database of handwriting and drawing specimens and proposed a method for selecting the most relevant information for diagnosing neurodegenerative disorders. The basic idea was to generate, for each handwriting sample, a color image to exploit the ability of convolutional neural network to automatically extract features from raw images. In the generated images, the color of each elementary trait encodes, in the three RGB channels, the dynamic information associated with that trait. Starting from the very encouraging obtained results, the aim of this study is twofold: On the one hand, we have tried to improve the feature extraction phase, associating further dynamic information with each handwritten trait. On the other hand, we have expanded the database of handwriting samples by adding specimen derived from more complex drawing tasks. Finally, we carried out a large set of experiments for comparing the results obtained by using standard online features with those obtained with our feature extraction approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. A secondary motor symptom experienced by some people with Parkinson’s disease, resulting in an abnormal small or cramped handwriting

References

  1. Vessio, G.: Dynamic handwriting analysis for neurodegenerative disease assessment: a literary review. Appl. Sci. 9(21), 4666 (2019)

    Article  Google Scholar 

  2. Kandel, E.R., Schwartz, J.H., Jessell, T.M.: Principles of Neural Science, 4th edn. McGraw-Hill Medical, New York (2000)

    Google Scholar 

  3. Lambert, J., Giffard, B., Nore, F., de la Sayette, V., Pasquier, F., Eustache, F.: Central and peripheral agraphia in alzheimer’s disease: from the case of auguste d. to a cognitive neuropsychology approach. Cortex 43(7), 935–951 (2007)

    Article  Google Scholar 

  4. Neils-Strunjas, J., Groves-Wright, K., Mashima, P., Harnish, S.: Dysgraphia in Alzheimer’s disease: a review for clinical and research purposes. J. Speech Lang. Hear. Res. 49(6), 1313–30 (2006)

    Article  Google Scholar 

  5. De Stefano, C., Fontanella, F., Impedovo, D., Pirlo, G., di Freca, A.S.: Handwriting analysis to support neurodegenerative diseases diagnosis: a review. Pattern Recognit. Lett. 121, 37–45 (2018)

    Article  Google Scholar 

  6. Werner, P., Rosenblum, S., Bar-On, G., Heinik, J., Korczyn, A.: Handwriting process variables discriminating mild alzheimer’s disease and mild cognitive impairment. J. Gerontol. Psychol. Sci. 61(4), 228–36 (2006)

    Article  Google Scholar 

  7. Cilia, N.D., De Stefano, C., Marrocco, C., Fontanella, F., Molinara, M., di Freca, A.S.: Deep transfer learning for alzheimer’s disease detection. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 9904–9911 (2021)

  8. LeCun, Y., Bengio, Y.: Convolutional Networks for Images, Speech, and Time-series. MIT Press, Cambridge (1995)

    Google Scholar 

  9. Cilia, N.D., De Stefano, C., Fontanella, F., Scotto di Freca, A.: An experimental protocol to support cognitive impairment diagnosis by using handwriting analysis. Proced. Comput. Sci. 141, 466–471 (2018)

    Article  Google Scholar 

  10. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: Proc. of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE Computer Society (2009)

  11. Cilia, N.D., De Stefano, C., Fontanella, F., Molinara, M., Di Freca, A.S.: Handwriting analysis to support alzheimer’s disease diagnosis: a preliminary study. In: Vento, M., Percannella, G. (eds.) Computer Analysis of Images and Patterns, pp. 143–151. Springer International Publishing, Cham (2019)

    Chapter  Google Scholar 

  12. Cilia, N.D., De Stefano, C., Fontanella, F., Molinara, M., Di Freca, A.S.: Using handwriting features to characterize cognitive impairment. In: Ricci, E., Rota Bulò, S., Snoek, C., Lanz, O., Messelodi, S., Sebe, N. (eds.) Image Analysis and Processing, pp. 683–693. Springer International Publishing, Cham (2019)

    Google Scholar 

  13. Cilia, N.D., De Stefano, C., Fontanella, F., di Freca, A.S.: How word choice affects cognitive impairment detection by handwriting analysis: A preliminary study. In: Cicirelli, F., Guerrieri, A., Pizzuti, C., Socievole, A., Spezzano, G., Vinci, A. (eds.) Artificial Life and Evolutionary Computation, pp. 113–123. Springer International Publishing, Cham (2020)

    Chapter  Google Scholar 

  14. Cilia, N.D., De Stefano, C., Fontanella, F., Scotto di Freca, A.: Using genetic algorithms for the prediction of cognitive impairments. In: Castillo, P.A. et al. (eds) Applications of Evolutionary Computation. EvoApplications 2020. Lecture Notes in Computer Science, vol. 12104, pp. 479–493. Springer, Cham (2020)

  15. Lei, B., Yang, M., Yang, P., Zhou, F., Hou, W., Zou, W., Li, X., Wang, T., Wang, S., Xiao, X.: Deep and joint learning of longitudinal data for alzheimer’s disease prediction. Pattern Recognition 107247 (2020)

  16. Cao, P., Liu, X., Yang, J., Zhao, D., Huang, M., Zaiane, O.: l2,1–l1 regularized nonlinear multi-task representation learning based cognitive performance prediction of alzheimer’s disease. Pattern Recognit. 79, 195–215 (2018)

    Article  Google Scholar 

  17. Zhang, Y., Zhang, H., Chen, X., Liu, M., Zhu, X., Lee, S.W., Shen, D.: Strength and similarity guided group-level brain functional network construction for MCI diagnosis. Pattern Recognit. 88, 421–430 (2018)

    Article  Google Scholar 

  18. Bi, X., Wang, H.: Early alzheimer’s disease diagnosis based on EEG spectral images using deep learning. Neural Netw. 114, 119–135 (2019)

    Article  Google Scholar 

  19. Fiscon, G., Weitschek, E., Cialini, A., Felici, G., Bertolazzi, P., De Salvo, S., Bramanti, A., Bramanti, P., De Cola, M.C.: Combining EEG signal processing with supervised methods for alzheimer’s patients classification. BMC Med. Inform. Decis. Mak. 18, 35 (2018)

    Article  Google Scholar 

  20. Bevilacqua, V., Loconsole, C., Brunetti, A., Cascarano, G.D., Lattarulo, A., Losavio, G., Di Sciascio, E.: A model-free computer-assisted handwriting analysis exploiting optimal topology ANNs on biometric signals in parkinson’s disease research. In: Huang, D.S. et al. (eds.) Intelligent Computing Theories and Application. ICIC 2018. Lecture Notes in Computer Science, vol. 10955, pp. 650–655. Springer, Cham (2018)

  21. Loconsole, C., Cascarano, G.D., Brunetti, A., Trotta, G.F., Losavio, G., Bevilacqua, V., Di Sciascio, E.: A model-free technique based on computer vision and sEMG for classification in parkinson’s disease by using computer-assisted handwriting analysis. Pattern Recognit. Lett. 121, 28–36 (2019)

    Article  Google Scholar 

  22. Diaz, M., Ferrer, M.A., Impedovo, D., Pirlo, G., Vessio, G.: Dynamically enhanced static handwriting representation for parkinson’s disease detection. Pattern Recognit. Lett. 128, 204–210 (2019)

    Article  Google Scholar 

  23. Diaz, M., Moetesum, M., Siddiqi, I., Vessio, G.: Sequence-based dynamic handwriting analysis for parkinson’s disease detection with one-dimensional convolutions and BiGRUs. Siddiqi 168, 114405 (2021)

    Google Scholar 

  24. El-Yacoubi, M.A., Garcia-Salicetti, S., Kahindo, C., Rigaud, A.S., Cristancho-Lacroix, V.: From aging to early-stage alzheimer’s: uncovering handwriting multimodal behaviors by semi-supervised learning and sequential representation learning. Pattern Recognit. 86, 112–133 (2019)

    Article  Google Scholar 

  25. Impedovo, D., Pirlo, G., Mangini, F.M., Barbuzzi, D., Rollo, A., Balestrucci, A., Impedovo, S., Sarcinella, L., O’Reilly, C., Plamondon, R.: Writing generation model for health care neuromuscular system investigation. In: Proceedings of CIBB 2013, pp. 137–148. Springer (2014)

  26. Pirlo, G., Cabrera, M.D., Ferrer-Ballester, M.A., Impedovo, D., Occhionero, F., Zurlo, U.: Early diagnosis of neurodegenerative diseases by handwritten signature analysis. In: ICIAP Workshops, pp. 290–297 (2015)

  27. Garre-Olmo, J., Faundez-Zanuy, M., de Ipiña, K.L., Calvo-Perxas, L., Turro-Garriga, O.: Kinematic and pressure features of handwriting and drawing: Preliminary results between patients with mild cognitive impairment, alzheimer disease and healthy controls. Curr. Alzheimer Res. 14, 1–9 (2017)

    Article  Google Scholar 

  28. Yan, J.H., Rountree, S., Massman, P., Smith Doody, R., Li, H.: Alzheimer’s disease and mild cognitive impairment deteriorate fine movement control. J. Psychiatr. Res. 42(14), 1203–1212 (2008)

    Article  Google Scholar 

  29. Schröter, A., Mergl, R., Bürger, K., Hampel, H., Möller, H.J., Hegerl, U.: Kinematic analysis of handwriting movements in patients with alzheimer’s disease, mild cognitive impairment, depression and healthy subjects. Dement. Geriatr. Cognit. Disord. 15(3), 132–42 (2003)

    Article  Google Scholar 

  30. Marcelli, A., Parziale, A., Santoro, A.: Modelling visual appearance of handwriting. In: Petrosino, A. (ed.) Image Analysis and Processing - ICIAP 2013. ICIAP 2013. Lecture Notes in Computer Science, vol. 8157, pp. 673–682. Springer, Berlin, Heidelberg (2013)

  31. Marcelli, A., Parziale, A., Senatore, R.: Some observations on handwriting from a motor learning perspective. In: 2nd International Workshop on Automated Forensic Handwriting Analysis (2013)

  32. Tseng, M.H., Cermak, S.A.: The influence of ergonomic factors and perceptual-motor abilities on handwriting performance. Am. J. Occup. Ther. 47(10), 919–926 (1993)

    Article  Google Scholar 

  33. Vyhnálek, M., Rubínová, E., Marková, H., Nikolai, T., Laczó, J., Andel, R., Hort, J.: Clock drawing test in screening for alzheimer’s dementia and mild cognitive impairment in clinical practice. Int. J. Geriatr. Psychiatry 32(9), 933–939 (2017)

    Article  Google Scholar 

  34. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Y. Bengio, Y. LeCun (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015). arxiv: abs/1409.1556

  35. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 770–778 (2016)

  36. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2818–2826 (2016)

  37. Szegedy, C., Ioffe, S., Vanhoucke, V.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), pp. 4278–4284. ACM (2016)

  38. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  39. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1-27:27 (2011)

    Article  Google Scholar 

  40. De Stefano, C., Fontanella, F., Marrocco, C., Di Freca, A.S.: A hybrid evolutionary algorithm for bayesian networks learning: An application to classifier combination. Appl. Evol. Comput. 6024, 221–230 (2010)

    Google Scholar 

  41. De Stefano, C., Fontanella, F., Folino, G., Di Freca, A.S.: A bayesian approach for combining ensembles of GP classifiers. Mult. Classif. Syst. MCS 6713, 26–35 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Information Engineering, University of Cassino and Southern Lazio, Cassino, FR, Italy

    Nicole D. Cilia, Tiziana D’Alessandro, Claudio De Stefano & Francesco Fontanella

  2. Institute for Computing and Information Sciences, Radboud University, Nijmegen, The Netherlands

    Nicole D. Cilia

Authors
  1. Nicole D. Cilia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tiziana D’Alessandro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claudio De Stefano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesco Fontanella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudio De Stefano.

Ethics declarations

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was partially supported by MIUR (Minister for Education, University and Research, Law 232/216, Department of Excellence)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cilia, N.D., D’Alessandro, T., De Stefano, C. et al. Deep transfer learning algorithms applied to synthetic drawing images as a tool for supporting Alzheimer’s disease prediction. Machine Vision and Applications 33, 49 (2022). https://doi.org/10.1007/s00138-022-01297-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-022-01297-8

Keywords

Search

Navigation