Skip to main content
SpringerLink
Log in

Neural Architecture Search in Graph Neural Networks

  • Conference paper
  • First Online:
Intelligent Systems (BRACIS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12319))

Included in the following conference series:

Abstract

Performing analytical tasks over graph data has become increasingly interesting due to the ubiquity and large availability of relational information. However, unlike images or sentences, there is no notion of sequence in networks. Nodes (and edges) follow no absolute order, and it is hard for traditional machine learning (ML) algorithms to recognize a pattern and generalize their predictions on this type of data. Graph Neural Networks (GNN) successfully tackled this problem. They became popular after the generalization of the convolution concept to the graph domain. However, they possess a large number of hyper-parameters and their design and optimization is currently hand-made, based on heuristics or empirical intuition. Neural Architecture Search (NAS) methods appear as an interesting solution to this problem. In this direction, this paper compares two NAS methods for optimizing GNN: one based on reinforcement learning and a second based on evolutionary algorithms. Results consider 7 datasets over two search spaces and show that both methods obtain similar accuracies to a random search, raising the question of how many of the search space dimensions are actually relevant to the problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In this work we use the terms “graph” and “network” interchangeably. When referring to “neural networks” we will use NN or “neural network”.

  2. 2.

    Code available at: https://github.com/mhnnunes/nas_gnn.

  3. 3.

    https://github.com/rusty1s/pytorch_geometric.

  4. 4.

    https://github.com/GraphNAS/GraphNAS.

  5. 5.

    We present only the results for the Macro search space because the results for Micro are very similar.

  6. 6.

    The smallest datasets (CIT and COR) are not present in the table because none of the generated architectures for these datasets exceeded GPU memory.

References

  1. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. TPAMI 39(12), 2481–2495 (2017)

    Google Scholar 

  2. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. JMLR 13, 281–305 (2012)

    Google Scholar 

  3. Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. In: Bengio, Y., LeCun, Y. (eds.) ICLR 2014 (2014)

    Google Scholar 

  4. Cai, H., Chen, T., Zhang, W., Yu, Y., Wang, J.: Efficient architecture search by network transformation. In: AAAI 2018 (2018)

    Google Scholar 

  5. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Lee, D.D., Sugiyama, M., von Luxburg, U., Guyon, I., Garnett, R. (eds.) NeurIPS 2016, pp. 3837–3845 (2016)

    Google Scholar 

  6. Eiben, A., Smith, J.: Introduction to Evolutionary Computing. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-44874-8

    Book  MATH  Google Scholar 

  7. Elshawi, R., Maher, M., Sakr, S.: Automated machine learning: state-of-the-art and open challenges. arXiv preprint arXiv:1906.02287 (2019)

  8. Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: a survey. JMLR 20, 55:1–55:21 (2019)

    Google Scholar 

  9. Gao, Y., Yang, H., Zhang, P., Zhou, C., Hu, Y.: Graph neural architecture search. In: IJCAI 2020, pp. 1403–1409 (2020)

    Google Scholar 

  10. Gori, M., Monfardini, G., Scarselli, F.: A new model for learning in graph domains. In: Proceedings, 2005 IEEE International Joint Conference on Neural Networks, vol. 2, pp. 729–734. IEEE (2005)

    Google Scholar 

  11. Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NIPS 2017 (2017)

    Google Scholar 

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR 2016, pp. 770–778 (2016)

    Google Scholar 

  13. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR 2017 (2017)

    Google Scholar 

  14. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  15. Pimenta, C.G., de Sá, A.G.C., Ochoa, G., Pappa, G.L.: Fitness landscape analysis of automated machine learning search spaces. In: Paquete, L., Zarges, C. (eds.) EvoCOP 2020. LNCS, vol. 12102, pp. 114–130. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43680-3_8

    Chapter  Google Scholar 

  16. Real, E., Aggarwal, A., Huang, Y., Le, Q.V.: Aging evolution for image classifier architecture search. In: AAAI 2019 (2019)

    Google Scholar 

  17. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. In: IEEE TNN 2009 (2009)

    Google Scholar 

  18. Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29(3), 93 (2008)

    Article  Google Scholar 

  19. Shchur, O., Mumme, M., Bojchevski, A., Günnemann, S.: Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018)

  20. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR 2018 (2018)

    Google Scholar 

  21. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. CoRR (2019)

    Google Scholar 

  22. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? In: ICLR 2019 (2019). OpenReview.net

  23. Zhang, M., Cui, Z., Neumann, M., Chen, Y.: An end-to-end deep learning architecture for graph classification. In: AAAI 2018 (2018)

    Google Scholar 

  24. Zhang, Z., Cui, P., Zhu, W.: Deep learning on graphs: a survey. In: TKDE 2020, p. 1 (2020)

    Google Scholar 

  25. Zhou, K., Song, Q., Huang, X., Hu, X.: Auto-GNN: neural architecture search of graph neural networks. arXiv preprint arXiv:1909.03184 (2019)

  26. Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: CVPR 2018 (2018)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank FAPEMIG (grant no. CEX-PPM-00098-17), MPMG (project Analytical Capabilities), CNPq (grant no. 310833/2019-1), CAPES, MCTIC/RNP (grant no. 51119) and H2020 (grant no. 777154) for the financial support.

Author information

Authors and Affiliations

  1. Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

    Matheus Nunes & Gisele L. Pappa

Authors
  1. Matheus Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gisele L. Pappa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matheus Nunes .

Editor information

Editors and Affiliations

  1. Federal University of São Carlos, São Carlos, Brazil

    Ricardo Cerri

  2. Federal University of ABC, Santo Andre, Brazil

    Ronaldo C. Prati

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nunes, M., Pappa, G.L. (2020). Neural Architecture Search in Graph Neural Networks. In: Cerri, R., Prati, R.C. (eds) Intelligent Systems. BRACIS 2020. Lecture Notes in Computer Science(), vol 12319. Springer, Cham. https://doi.org/10.1007/978-3-030-61377-8_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61377-8_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61376-1

  • Online ISBN: 978-3-030-61377-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation