In this section, the capacity of statistical machine learning techniques for recursive structure processing is investigated. While the universal approximation capability of recurrent and recursive networks for sequence and tree processing is well established, recent extensions to so-called contextual models have not yet been investigated in depth. Contextual models have been proposed to process acyclic graph structures. They rely on a restriction of the recurrence of standard models with respect to children of vertices as occurs e.g. in cascade correlation. This restriction allows to introduce recurrence with respect to parents of vertices without getting cyclic definitions. These models have very successfully been applied to various problems in computational chemistry. In this section, the principled information which can be processed in such a way and the approximation capabilities of realizations of this principle by means of neural networks are investigated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sperduti, A., Starita, A.: Supervised neural networks for the classification of structures. IEEE Trans. Neural Networks 8(3) (1997) 714-735
Micheli, A., Portera, F., Sperduti, A.: A preliminary empirical comparison of recursive neural networks and tree kernel methods on regression tasks for tree structured domains. Neurocomputing 64 (2005) 73-92
Bianucci, A., Micheli, A., Sperduti, A., Starita, A.: Quantitative structure-activity relationships of benzodiazepines by Recursive Cascade Correlation. In IEEE, ed.: Proceedings of IJCNN ’98 - IEEE World Congress on Computational Intelligence, Anchorage, Alaska (1998) 117-122
Bianucci, A.M., Micheli, A., Sperduti, A., Starita, A.: Application of cascade correlation networks for structures to chemistry. Journal of Applied Intelligence (Kluwer Academic Publishers) 12 (2000) 117-146
Micheli, A., Sperduti, A., Starita, A., Bianucci, A.M.: Analysis of the internal representations developed by neural networks for structures applied to quan-titative structure-activity relationship studies of benzodiazepines. Journal of Chem. Inf. and Comp. Sci. 41(1) (2001) 202-218
Micheli, A., Sperduti, A., Starita, A., Bianucci, A.M.: Design of new biologically active molecules by recursive neural networks. In: IJCNN’2001 - Proceedings of the INNS-IEEE International Joint Conference on Neural Networks, Washington, DC (2001) 2732-2737
Bernazzani, L., Duce, C., Micheli, A., Mollica, V., Sperduti, A., Starita, A., Tiné, M.R.: Predicting physical-chemical properties of compounds from molec-ular structures by recursive ne ural networks. J. Chem. Inf. Model 46(5) (2006) 2030-2042
Duce, C., Micheli, A., Solaro, R., Starita, A., Tiné, M.R.: Prediction of chemical- physical properties by neural networks for structures. Macromolecular Symposia 234(1) (2006) 13-19
Duce, C., Micheli, A., Starita, A., Tiné, M.R., Solaro, R.: Prediction of polymer properties from their structure by recursive neural networks. Macromolecular Rapid Communications 27(9) (2006) 711-715
Micheli, A., Sona, D., Sperduti, A.: Contextual processing of structured data by recursive cascade correlation. IEEE Trans. Neural Networks 15(6) (2004) 1396-1410
Hammer, B.: Learning with Recurrent Neural Networks. Volume 254 of Springer Lecture Notes in Control and Information Sciences. Springer-Verlag (2000)
Elman, J.L.: Finding structure in time. Cognitive Science 14 (1990) 179-211
Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In Touretzky, D., ed.: Advances in Neural Information Processing Systems 2, San Mateo, CA: Morgan Kaufmann (1990) 524-532
Fahlman, S.E.: The recurrent cascade-correlation architecture. In Lippmann, R., Moody, J., Touretzky, D., eds.: Advances in Neural Information Processing Systems 3, San Mateo, CA, Morgan Kaufmann Publishers (1991) 190-196
Sperduti, A.: On the computational power of recurrent neural networks for structures. Neural Networks 10(3) (1997) 395-400
Carrasco, R., Forcada, M.: Simple strategies to encode tree automata in sigmoid recursive neural networks. IEEE TKDE 13(2) (2001) 148-156
Gori, M., Kuchler, A., Sperduti, A.: On the implementation of frontier-to-root tree automata in recursive neural networks. IEEE Transactions on Neural Networks 10(6) (1999) 1305 -1314
Hammer, B.: On the learnability of recursive data. Mathematics of Control Signals and Systems 12 (1999) 62-79
Hammer, B.: Generalization ability of folding networks. IEEE TKDE 13(2) (2001) 196-206
Micheli, A., Sona, D., Sperduti, A.: Bi-causal recurrent cascade correlation. In: IJCNN’2000 - Proceedings of the IEEE-INNS-ENNS International Joint Con-ference on Neural Networks. Volume 3. (2000) 3-8
Micheli, A., Sona, D., Sperduti, A.: Recursive cascade correlation for contex-tual processing of structured data. In: Proc. of the Int. Joint Conf. on Neural Networks - WCCI-IJCNN’2002. Volume 1. (2002) 268-273
Hammer, B., Micheli, A., Sperduti, A.: Universal approximation capability of cascade correlation for structures. Neural Computation 17(5) (2005) 1109-1159
Kilian, J., Siegelmann, H.T.: The dynamic universality of sigmoidal neural networks. Information and Computation 128 (1996)
Siegelmann, H.T.: The simple dynamics of super Turing theories. Theoretical Computer Science 168 (1996)
Siegelmann, H.T., Sontag, E.D.: Analog computation, neural networks, and circuits. Theoretical Computer Science 131 (1994)
Siegelmann, H.T., Sontag, E.D.: On the computational power of neural net-works. Journal of Computer and System Sciences 50 (1995)
Omlin, C., Giles, C.: Stable encoding of large finite-state automata in recurrent neural networks with sigmoid discriminants. Neural Computation 8 (1996)
Frasconi, P., Gori, M., Kuechler, A., Sperduti, A.: From sequences to data structures: Theory and applications. In Kolen, J., Kremer, S., eds.: A Field Guide to Dynamic Recurrent Networks. IEEE Press (2001) 351-374
Frasconi, P., Gori, M.: Computational capabilities of local-feedback recurrent networks acting as finite-state machines. IEEE Transactions on Neural Networks 7 (6) (1996)1521-1524
Giles, C., Chen, D., Sun, G., Chen, H., Lee, Y., Goudreau, M.: Constructive learning of recurrent neural networks: limitations of recurrent cascade correla-tion and a simple solution. IEEE Transactions on Neural Networks 6(4) (1995) 829-836
K., F., Nakamura, Y.: Approximation of dynamical systems by continuous time recurrent neural networks. Neural Networks 6(6) (1993) 801-806
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks (1989) 359-366
Bianucci, A.M., Micheli, A., Sperduti, A., Starita, A.: A novel approach to QSPR/QSAR based on neural networks for structures. In Sztandera, L.M., Cartwright, H.M., eds.: Soft Computing Approaches in Chemistry. Springer-Verlag, Heidelberg (2003) 265-296
Micheli, A.: Recursive Processing of Structured Domains in Machine Learning. Ph.d. thesis: TD-13/03, Department of Computer Science, University of Pisa, Pisa, Italy (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hammer, B., Micheli, A., Sperduti, A. (2007). Adaptive Contextual Processing of Structured Data by Recursive Neural Networks: A Survey of Computational Properties. In: Hammer, B., Hitzler, P. (eds) Perspectives of Neural-Symbolic Integration. Studies in Computational Intelligence, vol 77. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73954-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-73954-8_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73953-1
Online ISBN: 978-3-540-73954-8
eBook Packages: EngineeringEngineering (R0)