Abstract
Kernel-based learning models such as support vector machines (SVMs) can seamlessly deal with graph structures thanks to suitable kernel functions that compute a particular similarity between pairs of data samples. In the last two decades, a plethora of graph kernels have been proposed, motivated by theoretical properties or designed specifically for an application domain. Computing graph kernels however presents a significant cost for both training and inference, since predictions on unseen graphs require evaluating the kernel e.g. between the new sample and all support vectors, and solutions to an SVM optimization problem are not guaranteed to be sparse. In this paper, we present a method to select a minimum set of spanning vectors for the solutions of SVMs, based on the rank-revealing QR decomposition of the kernel Gram matrix. We apply it on 18 real-world classification tasks on chemical compounds, showing its effectiveness to reduce the computational burden in performing inference on unseen graphs by minimizing the number of support vectors without penalizing accuracy. This in turn gives us a tool to better analyze the trade-off between accuracy, expressiveness and inference cost among different graph kernels.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bacciu, D., Errica, F., Micheli, A., Podda, M.: A gentle introduction to deep learning for graphs. Neural Netw. 129, 203–221 (2020). https://doi.org/10.1016/j.neunet.2020.06.006
Borgwardt, K., Ghisu, E., Llinares-López, F., O’Bray, L., Rieck, B.: Graph kernels: state-of-the-art and future challenges. Found. Trends Mach. Learn. 13(5–6), 531–712 (2020). https://doi.org/10.1561/2200000076
Borgwardt, K.M., Ong, C.S., Schonauer, S., Vishwanathan, S.V.N., Smola, A.J., Kriegel, H.P.: Protein function prediction via graph kernels. Bioinformatics 21(Suppl. 1), i47–i56 (2005). https://doi.org/10.1093/bioinformatics/bti1007
Borgwardt, K., Kriegel, H.: Shortest-path kernels on graphs. In: Proceedings of the Fifth IEEE International Conference on Data Mining, pp. 74–81 (2005). https://doi.org/10.1109/ICDM.2005.132
Chandrasekaran, S., Ipsen, I.C.F.: On rank-revealing factorisations. SIAM J. Matrix Anal. Appl. 15(2), 592–622 (1994). https://doi.org/10.1137/S0895479891223781
Downs, T., Gates, K., Masters, A.: Exact simplification of support vector solutions. J. Mach. Learn. Res. 2(Dec), 293–297 (2001). https://www.jmlr.org/papers/v2/downs01a.html
Gu, M., Eisenstat, S.C.: Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM J. Sci. Comput. 17(4), 848–869 (1996). https://doi.org/10.1137/0917055
Hong, Y.P., Pan, C.T.: Rank-revealing QR factorizations and the singular value decomposition. Math. Comput. 58(197), 213–232 (1992). https://doi.org/10.2307/2153029
Joachims, T.: Making large-scale support vector machine learning practical. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods: Support Vector Learning, Chapter 11, pp. 169–184 (1999). https://doi.org/10.7551/mitpress/1130.003.0015
Kriege, N.M., Johansson, F.D., Morris, C.: A survey on graph kernels. Appl. Netw. Sci. 5(1), 1–42 (2019). https://doi.org/10.1007/s41109-019-0195-3
Lee, Y.J., Mangasarian, O.L.: RSVM: reduced support vector machines. In: Proceedings of the 2001 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, Philadelphia, PA (2001). https://doi.org/10.1137/1.9781611972719.13
Morris, C., Kriege, N.M., Bause, F., Kersting, K., Mutzel, P., Neumann, M.: TUDataset: a collection of benchmark datasets for learning with graphs. In: ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020) (2020). https://www.graphlearning.io
Oneto, L., Navarin, N., Donini, M., Sperduti, A., Aiolli, F., Anguita, D.: Measuring the expressivity of graph kernels through statistical learning theory. Neurocomputing 268, 4–16 (2017). https://doi.org/10.1016/j.neucom.2017.02.088
Pan, C.T., Tang, P.T.P.: Bounds on singular values revealed by QR factorizations. BIT Numer. Math. 39(4), 740–756 (1999). https://doi.org/10.1023/A:1022395308695
Panja, R., Pal, N.R.: MS-SVM: minimally spanned support vector machine. Appl. Soft Comput. J. 64, 356–365 (2018). https://doi.org/10.1016/j.asoc.2017.12.017
Ralaivola, L., Swamidass, S.J., Saigo, H., Baldi, P.: Graph kernels for chemical informatics. Neural Netw. 18(8), 1093–1110 (2005). https://doi.org/10.1016/j.neunet.2005.07.009
Schölkopf, B., Smola, A.J., Williamson, R.C., Bartlett, P.L.: New support vector algorithms. Neural Comput. 12(5), 1207–1245 (2000). https://doi.org/10.1162/089976600300015565
Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge (2001). https://doi.org/10.7551/mitpress/4175.001.0001
Shervashidze, N., Schweitzer, P., van Leeuwen, E.J., Mehlhorn, K., Borgwardt, K.M.: Weisfeiler-Lehman graph kernels. J. Mach. Learn. Res. 12, 2539–2561 (2011). https://doi.org/10.5555/1953048.2078187
Shervashidze, N., Vishwanathan, S.V., Petri, T.H., Mehlhorn, K., Borgwardt, K.M.: Efficient graphlet kernels for large graph comparison. In: Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, vol. 5, pp. 488–495 (2009). https://proceedings.mlr.press/v5/shervashidze09a.html
Siglidis, G., Nikolentzos, G., Limnios, S., Giatsidis, C., Skianis, K., Vazirgiannis, M.: GraKeL: a graph kernel library in Python. J. Mach. Learn. Res. 21(54), 1–5 (2020). http://jmlr.org/papers/v21/18-370.html
Tran, Q.A., Zhang, Q.L., Li, X.: Reduce the number of support vectors by using clustering techniques. In: Proceedings of the Second International Conference on Machine Learning and Cybernetics, vol. 2, pp. 1245–1248 (2003). https://doi.org/10.1109/icmlc.2003.1259678
Yi, H.C., You, Z.H., Huang, D.S., Kwoh, C.K.: Graph representation learning in bioinformatics: trends, methods and applications. Brief. Bioinform. 23(1), 1–16 (2022). https://doi.org/10.1093/bib/bbab340
Acknowledgement
Research partly funded by PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000013 - “FAIR - Future Artificial Intelligence Research” - Spoke 1 “Human-centered AI”, funded by the European Commission under the NextGeneration EU programme.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tortorella, D., Micheli, A. (2023). Minimum Spanning Set Selection in Graph Kernels. In: Vento, M., Foggia, P., Conte, D., Carletti, V. (eds) Graph-Based Representations in Pattern Recognition. GbRPR 2023. Lecture Notes in Computer Science, vol 14121. Springer, Cham. https://doi.org/10.1007/978-3-031-42795-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-42795-4_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42794-7
Online ISBN: 978-3-031-42795-4
eBook Packages: Computer ScienceComputer Science (R0)
