COSNet: A Cost Sensitive Neural Network for Semi-supervised Learning in Graphs

  • Alberto Bertoni
  • Marco Frasca
  • Giorgio Valentini
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6911)


The semi-supervised problem of learning node labels in graphs consists, given a partial graph labeling, in inferring the unknown labels of the unlabeled vertices. Several machine learning algorithms have been proposed for solving this problem, including Hopfield networks and label propagation methods; however, some issues have been only partially considered, e.g. the preservation of the prior knowledge and the unbalance between positive and negative labels. To address these items, we propose a Hopfield-based cost sensitive neural network algorithm (COSNet). The method factorizes the solution of the problem in two parts: 1) the subnetwork composed by the labelled vertices is considered, and the network parameters are estimated through a supervised algorithm; 2) the estimated parameters are extended to the subnetwork composed of the unlabeled vertices, and the attractor reached by the dynamics of this subnetwork allows to predict the labeling of the unlabeled vertices. The proposed method embeds in the neural algorithm the “a priori” knowledge coded in the labelled part of the graph, and separates node labels and neuron states, allowing to differentially weight positive and negative node labels. Moreover, COSNet introduces an efficient cost-sensitive strategy which allows to learn the near-optimal parameters of the network in order to take into account the unbalance between positive and negative node labels. Finally, the dynamics of the network is restricted to its unlabeled part, preserving the minimization of the overall objective function and significantly reducing the time complexity of the learning algorithm. COSNet has been applied to the genome-wide prediction of gene function in a model organism. The results, compared with those obtained by other semi-supervised label propagation algorithms and supervised machine learning methods, show the effectiveness of the proposed approach.


Label Propagation Node Label Protein Function Prediction Energy Global Minimum Gene Function Prediction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zheleva, E., Getoor, L., Sarawagi, S.: Higher-order graphical models for classification in social and affiliation networks. In: NIPS 2010 Workshop on Networks Across Disciplines: Theory and Applications, Whistler BC, Canada (2010)Google Scholar
  2. 2.
    Mostafavi, S., Morris, Q.: Fast integration of heterogeneous data sources for predicting gene function with limited annotation. Bioinformatics 26(14), 1759–1765 (2010)CrossRefGoogle Scholar
  3. 3.
    Vazquez, A., et al.: Global protein function prediction from protein-protein interaction networks. Nature Biotechnology 21, 697–700 (2003)CrossRefGoogle Scholar
  4. 4.
    Leskovec, J., et al.: Statistical properties of community structure in large social and information networks. In: Proc. 17th Int. Conf. on WWW, pp. 695–704. ACM, New York (2008)Google Scholar
  5. 5.
    Bilgic, M., Mihalkova, L., Getoor, L.: Active learning for networked data. In: Proc. of the 27th ICML, Haifa, Israel (2010)Google Scholar
  6. 6.
    Marcotte, E., et al.: A combined algorithm for genome-wide prediction of protein function. Nature 402, 83–86 (1999)CrossRefGoogle Scholar
  7. 7.
    Oliver, S.: Guilt-by-association goes global. Nature 403, 601–603 (2000)CrossRefGoogle Scholar
  8. 8.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning with gaussian fields and harmonic functions. In: Proc. of the 20th ICML, Washintgton DC, USA (2003)Google Scholar
  9. 9.
    Zhou, D.: et al.: Learning with local and global consistency. In: Adv. Neural Inf. Process. Syst., vol. 16, pp. 321–328 (2004)Google Scholar
  10. 10.
    Szummer, M., Jaakkola, T.: Partially labeled classification with markov random walks. In: NIPS 2001, Whistler BC, Canada, vol. 14 (2001)Google Scholar
  11. 11.
    Azran, A.: The rendezvous algorithm: Multi- class semi-supervised learning with Markov random walks. In: Proc. of the 24th ICML (2007)Google Scholar
  12. 12.
    Belkin, M., Matveeva, I., Niyogi, P.: Regularization and semi-supervised learning on large graphs. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS (LNAI), vol. 3120, pp. 624–638. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  13. 13.
    Delalleau, O., Bengio, Y., Le Roux, N.: Efficient non-parametric function induction in semi-supervised learning. In: Proc. of the Tenth Int. Workshop on Artificial Intelligence and Statistics (2005)Google Scholar
  14. 14.
    Belkin, M., Niyogi, P.: Using manifold structure for partially labeled classification. In: Adv. Neural Inf. Process. Syst., vol. 15 (2003)Google Scholar
  15. 15.
    Nabieva, E., et al.: Whole-proteome prediction of protein function via graph-theoretic analysis of interaction maps. Bioinformatics 21(S1), 302–310 (2005)CrossRefGoogle Scholar
  16. 16.
    Deng, M., Chen, T., Sun, F.: An integrated probabilistic model for functional prediction of proteins. J. Comput. Biol. 11, 463–475 (2004)CrossRefGoogle Scholar
  17. 17.
    Tsuda, K., Shin, H., Scholkopf, B.: Fast protein classification with multiple networks. Bioinformatics 21(suppl 2), ii59–ii65 (2005)CrossRefGoogle Scholar
  18. 18.
    Mostafavi, S., et al.: GeneMANIA: a real-time multiple association network integration algorithm for predicting gene function. Genome Biology 9(S4) (2008)Google Scholar
  19. 19.
    Hopfield, J.: Neural networks and physical systems with emergent collective compautational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Bengio, Y., Delalleau, O., Le Roux, N.: Label Propagation and Quadratic Criterion. In: Chapelle, O., Scholkopf, B., Zien, A. (eds.) Semi-Supervised Learning, pp. 193–216. MIT Press, Cambridge (2006)Google Scholar
  21. 21.
    Karaoz, U., et al.: Whole-genome annotation by using evidence integration in functional-linkage networks. Proc. Natl Acad. Sci. USA 101, 2888–2893 (2004)CrossRefGoogle Scholar
  22. 22.
    Ruepp, A., et al.: The FunCat, a functional annotation scheme for systematic classification of proteins from whole genomes. Nucleic Acids Research 32(18), 5539–5545 (2004)CrossRefGoogle Scholar
  23. 23.
    Wang, D.: Temporal pattern processing. In: The Handbook of Brain Theory and Neural Networks, pp. 1163–1167 (2003)Google Scholar
  24. 24.
    Liu, H., Hu, Y.: An application of hopfield neural network in target selection of mergers and acquisitions. In: International Conference on Business Intelligence and Financial Engineering, pp. 34–37 (2009)Google Scholar
  25. 25.
    Zhang, F., Zhang, H.: Applications of a neural network to watermarking capacity of digital image. Neurocomputing 67, 345–349 (2005)CrossRefGoogle Scholar
  26. 26.
    Tsirukis, A.G., Reklaitis, G.V., Tenorio, M.F.: Nonlinear optimization using generalized hopfield networks. Neural Comput. 1, 511–521 (1989)CrossRefGoogle Scholar
  27. 27.
    Ashburner, M., et al.: Gene ontology: tool for the unification of biology. the gene ontology consortium. Nature Genetics 25(1), 25–29 (2000)CrossRefGoogle Scholar
  28. 28.
    Agresti, A., Coull, B.A.: Approximate is better than exact for interval estimation of binomial proportions. Statistical Science 52(2), 119–126 (1998)MathSciNetGoogle Scholar
  29. 29.
    Brown, L.D., Cai, T.T., Dasgupta, A.: Interval estimation for a binomial proportion. Statistical Science 16, 101–133 (2001)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Cesa-Bianchi, N., Valentini, G.: Hierarchical cost-sensitive algorithms for genome-wide gene function prediction. Journal of Machine Learning Research, W&C Proceedings, Machine Learning in Systems Biology 8, 14–29 (2010)Google Scholar
  31. 31.
    Eddy, S.R.: Profile hidden Markov models. Bioinformatics 14(9), 755–763 (1998)CrossRefGoogle Scholar
  32. 32.
    Spellman, P.T., et al.: Comprehensive identification of cell cycle-regulated genes of the yeast saccharomyces cerevisiae by microarray hybridization. Molecular Biology of the Cell 9(12), 3273–3297 (1998)CrossRefGoogle Scholar
  33. 33.
    Gasch, P., et al.: Genomic expression programs in the response of yeast cells to environmental changes. Mol. Biol. Cell 11(12), 4241–4257 (2000)CrossRefGoogle Scholar
  34. 34.
    Stark, C., et al.: Biogrid: a general repository for interaction datasets. Nucleic Acids Research 34(Database issue), 535–539 (2006)CrossRefGoogle Scholar
  35. 35.
    von Mering, C., et al.: Comparative assessment of large-scale data sets of protein-protein interactions. Nature 417(6887), 399–403 (2002)CrossRefGoogle Scholar
  36. 36.
    Chua, H., Sung, W., Wong, L.: An efficient strategy for extensive integration of diverse biological data for protein function prediction. Bioinformatics 23(24), 3364–3373 (2007)CrossRefGoogle Scholar
  37. 37.
    Lin, H.T., Lin, C.J., Weng, R.: A note on platt’s probabilistic outputs for support vector machines. Machine Learning 68(3), 267–276 (2007)CrossRefGoogle Scholar
  38. 38.
    Brown, M.P.S., et al.: Knowledge-based analysis of microarray gene expression data by using support vector machines. Proceedings of the National Academy of Sciences of the United States of America 97(1), 267–276 (2000)MathSciNetGoogle Scholar
  39. 39.
    Pavlidis, P., et al.: Learning gene functional classifications from multiple data types. Journal of Computational Biology 9, 401–411 (2002)CrossRefGoogle Scholar
  40. 40.
    Wilcoxon, F.: Individual comparisons by ranking methods. Journal of Computational Biology 1(6), 80–83 (1945)Google Scholar
  41. 41.
    Re, M., Valentini, G.: Simple ensemble methods are competitive with state-of-the-art data integration methods for gene function prediction. Journal of Machine Learning Research, W&C Proceedings, Machine Learning in Systems Biology 8, 98–111 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Alberto Bertoni
    • 1
  • Marco Frasca
    • 1
  • Giorgio Valentini
    • 1
  1. 1.DSI, Dipartimento di Scienze dell’ InformazioneUniversità degli Studi di MilanoMilanoItalia

Personalised recommendations