Artificial Neural Networks Training Acceleration Through Network Science Strategies

  • Lucia CavallaroEmail author
  • Ovidiu Bagdasar
  • Pasquale De Meo
  • Giacomo Fiumara
  • Antonio Liotta
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11974)


Deep Learning opened artificial intelligence to an unprecedented number of new applications. A critical success factor is the ability to train deeper neural networks, striving for stable and accurate models. This translates into Artificial Neural Networks (ANN) that become unmanageable as the number of features increases. The novelty of our approach is to employ Network Science strategies to tackle the complexity of the actual ANNs at each epoch of the training process. The work presented herein originates in our earlier publications, where we explored the acceleration effects obtained by enforcing, in turn, scale freeness, small worldness, and sparsity during the ANN training process. The efficiency of our approach has also been recently confirmed by independent researchers, who managed to train a million-node ANN on non-specialized laptops. Encouraged by these results, we have now moved into having a closer look at some tunable parameters of our previous approach to pursue a further acceleration effect. We now investigate on the revise fraction parameter, to verify the necessity of the role of its double-check. Our method is independent of specific machine learning algorithms or datasets, since we operate merely on the topology of the ANNs. We demonstrate that the revise phase can be avoided in order to half the overall execution time with an almost negligible loss of quality.


Network Science Artificial Neural Networks 



We thank Dr Decebal Costantin Mocanu for providing constructive feedback.


  1. 1.
    Barabási, A.L., Pósfai, M.: Network Science. Cambridge University Press, Cambridge (2016). Scholar
  2. 2.
    Berman, D.S., Buczak, A., Chavis, J., Corbett, C.: A survey of deep learning methods for cyber security. Information 10, 122 (2019). Scholar
  3. 3.
    Cao, C., et al.: Deep learning and its applications in biomedicine. Genomics Proteomics Bioinform. 16(1), 17–32 (2018). Scholar
  4. 4.
    Chen, H., Engkvist, O., Wang, Y., Olivecrona, M., Blaschke, T.: The rise of deep learning in drug discovery. Drug Discov. Today 23(6), 1241–1250 (2018). Scholar
  5. 5.
    Erdös, P., Rényi, A.: On random graphs I. Publ. Math. Debr. 6, 290–297 (1959)zbMATHGoogle Scholar
  6. 6.
    Gardner, M., Dorling, S.: Artificial neural networks (the multilayer perceptron)–a review of applications in the atmospheric sciences. Atmos. Environ. 32(14–15), 2627–2636 (1998)CrossRefGoogle Scholar
  7. 7.
    Goodfellow, I.J., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.orgzbMATHGoogle Scholar
  8. 8.
    Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29(6), 82–97 (2012). Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017). Scholar
  10. 10.
    Latora, V., Nicosia, V., Russo, G.: Complex Networks: Principles, Methods and Applications. Cambridge University Press, Cambridge (2017)CrossRefGoogle Scholar
  11. 11.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nat. Cell Biol. 521(7553), 436–444 (2015). Scholar
  12. 12.
    Liu, S., Mocanu, D.C., Matavalam, A., Pei, Y., Pechenizkiy, M.: Sparse evolutionary deep learning with over one million artificial neurons on commodity hardware. arXiv:1901.09181 (2019)
  13. 13.
    Mocanu, D.C., Mocanu, E., Stone, P., Nguyen, P., Gibescu, M., Liotta, A.: Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nat. Commun. 9, 2383 (2018). Scholar
  14. 14.
    Ruano-Ordás, D., Yevseyeva, I., Fernandes, V.B., Méndez, J.R., Emmerich, M.T.M.: Improving the drug discovery process by using multiple classifier systems. Expert Syst. Appl. 121, 292–303 (2019). Scholar
  15. 15.
    Yu, D., Deng, L.: Deep learning and its applications to signal and information processing [exploratory DSP]. IEEE Signal Process. Mag. 28(1), 145–154 (2011). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of DerbyDerbyUK
  2. 2.Polo Universitario AnnunziataUniversity of MessinaMessinaItaly
  3. 3.MIFT DepartmentUniversity of MessinaMessinaItaly
  4. 4.Edinburgh Napier UniversityEdinburghUK

Personalised recommendations