Abstract
Connectionist methods and knowledge-based techniques are two largely complementary approaches to natural language processing (NLP). However, they both have some potential problems which preclude their being a general purpose processing method. Research reveals that a hybrid processing approach that combines connectionist with symbolic techniques may be able to use the strengths of one processing paradigm to address the weakness of the other one. Hence, a system that effectively combines the two different approaches can be superior to either one in isolation.
This paper describes a hybrid system—SYMCON (SYMbolic and CONnectionist) which integrates symbolic and connectionist techniques in an attempt to solve the problem of word sense disambiguation (WSD), which is arguably one of the most fundamental and difficult issues in NLP. It consists of three sub-systems: first, a distributed simple recurrent network (SRN) is trained by using the standard back-propagation algorithm to learn the semantic relationships among concepts, thereby generating categorical constraints that are supplied to the other two sub-systems as the initial results of pre-processing. The second sub-system of SYMCON is a knowledge-based symbolic component consisting of a knowledge base containing general inferencing rules in a certain application domain. Third, a localist network is used to select the best interpretation among multiple alternatives and potentially ambiguous inference paths by spreading activation throughout the network. The structure, initial states, and connection weights of the network are determined by the processing outcome in the other two sub-systems. This localist network can be viewed as a medium between the distributed network and the symbolic sub-system. Such a hybrid symbolic/connectionist system combines information from all three sources to select the most plausible interpretation for ambiguous words.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H. Alshawi, B. Boguraev, and T. Briscoe, “Towards a dictionary support environment for real time parsing,” Proc. of the 2nd European ACL Conf., 1985, pp. 171–178.
E. Charniak, “Passing markers: A theory of contextual influence in language comprehension,” Cognitive Science, vol. 7, pp. 171–190, 1983.
Collins Concise School Dictionary, William Collins Sons and Co. Ltd., 1992
G.W. Cottrell, A Connectionist Approach to Word Sense Disambiguation, Morgan Kaufmann Publishers, 1989.
M.G. Dyer, In-depth Understand: A Computer Model of Integrated Processing for Narrative Comprehension, Cambridge, MA: MIT Press, 1983.
J.L. Elman, “Finding structure in time,” Cognitive Science, vol. 14, pp. 179–211, 1990.
J.L. Elman, “Distributed representations, simple recurrent networks, and grammatical structure,” Machine Learning, vol. 7, pp. 195–225, 1991.
L. Fu, “Integration of neural heuristics into knowledge-based inference,” Connection Science, vol. 1,no. 3, pp. 325–340, 1989.
L. Fu, “A Parallel distributed approach for knowledge-based inference and learning,” in Hybrid Architecture for Intelligence Systems edited by A. Kandel, and G. Langholz, CRC Press, pp. 88–103, 1992.
L. Fu, Neural Networks in Computer Intelligence, McGraw Hill, 1994.
W.A. Gale, K. W. Church, and D. Yarowsky, “A method for disambiguating word sense in a large corpus,” Computers and the Humanities, vol. 26,no. 5–6, pp. 415–439, 1992.
S. Ganatilake and S. Khebbal, Intelligent Hybrid Systems, Wiley and Sons, 1995.
C.M. Guo, “Deriving a natural set of semantic primitives from Longman Dictionary of Contemporary English,” Proc. of the 2nd Irish Conf. on AI and Cognitive Science, Dublin, 1989, pp. 218–227.
J.A. Hendler, “Marker passing over microfeatures: Towards a hybrid symbolic/connectionist model,” Cognitive Science, vol. 13, pp. 79–106, 1989.
G. Hirst, Semantic Interpretation and the Resolution of Ambiguity, Cambridge University Press: Cambridge, 1987.
V. Honavar and L. Uhr (eds.), Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, Academic Press, 1994.
A.H. Kawamoto, “Distributed representations of ambiguous words and their resolution in a connectionist network,” Lexical Ambiguity Resolution, edited by S.L. Small, Morgan Kaufmann, pp. 195–228, 1988.
T.E. Lange and M.G. Dyer, “High-level inferencing in a connectionist network,” Connection Science, vol. 1,no. 2, pp. 181–217, 1989.
J.L. McClelland and D.E. Rumelhart, “An interactive activation model of context effects in letter perception: Part I. An account of basic findings,” Psychological Review, vol. 88, pp. 375–407, 1981.
L.R. Medsker, Hybrid Neural Network and Expert Systems, Kluwer Academic Publishers, 1994.
E. Riloff and W. Lehnert, “Classifying texts using relevancy signatures,” Proc. of AAAI, 1992, pp. 329–334.
N.E. Sharkey, “Connectionist representation techniques,” Artificial Intelligence Review, vol. 5, pp. 143–167, 1991.
L. Shastri, “A connectionist approach to knowledge representation and limited inference,” Cognitive Science, vol. 12, pp. 331–392, 1988.
S. Stevenson, “A parallel constraint satisfaction and spreading activation model for revolving syntactic ambiguity,” Proc. of the 12th Annual Conf. of Cognitive Science Society, USA, pp. 396–403, 1990.
R. Sun and L.A. Bookman (eds.), Computational Architectures Integrating Neural and Symbolic Processes, A Perspective on the State of the Art, Kluwer Academic Publishers, 1995.
R.F.E. Sutcliffe, “Constructing distributed semantic sexical representations using a machine readable dictionary,” AI and Cognition Science 92, edited by K. Ryan and R.F.E. Sutcliffe, Springer-Verlag, pp. 210–223, 1992.
G. Towell, “Symbolic knowledge and neural networks: insertion, refinement, and extraction,” Ph.D. thesis, Computer Science Department, University of Wisconsin, Madison, WI, 1991.
G. Towell and J. Shavlik, “Knowledge-based artificial neural networks,” Artificial Intelligence, vol. 70, pp. 119–166, 1994.
D. Waltz and J.B. Pollack, “Massively parallel parsing: A strongly interactive model of natural language interpretation,” Cognitive Science, vol. 9, pp. 51–74, 1985.
S. Wermter and W.G. Lehnert, “A hybrid symbolic/connectionist model for noun phrase understanding,” Connection Science, vol. 1,no. 3, pp. 225–272, 1989.
Y. Wilks, D. Fass, and C.M. Guo et al., “Providing machine tractable dictionary tools,” Machine Translation, no. 5, pp. 99–154, 1990.
X. Wu, M. McTear, and P.C. Ojha, “Pronoun disambiguation using a hybrid system,” Proc. of 14th Int. Conf. on AI, KBS, Expert System and Natural Language Processing, Paris, 1994, vol. 3, pp. 67–76.
X. Wu, M. McTear, and P.C. Ojha, “Learning semantic relationships and syntactic roles in a simple recurrent network,” Proc. of Int. Conf. on New Methods in Language Processing, UMIST, Manchester, 1994, pp. 23–29.
X. Wu, M. McTear, and P.C. Ojha, “A knowledge-based hybrid approach to word sense disambiguation,” Proc. of Int. Workshop on Knowledge Engineering and Applications, PRICAI'94, Beijing, 1994, pp. 71–77.
X. Wu, M. McTear, and P.C. Ojha, “A hybrid system framework for disambiguating word senses,” Proc. of the IJCAI'95 Workshop on Connectionist and Symbolic Integration, Montreal, 1995, pp. 56–62.
X. Wu, M. McTear, and P.C. Ojha, “A representational scheme for a hybrid natural language processing system,” Proc. of the IEEE Int. Conf. on Systems, Man and Cybernetics, Vancouver, Oct. 1995, vol. 4, pp. 3162–3167.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wu, X., Mctear, M. & Ojha, P. SYMCON—A Hybrid Symbolic/Connectionist System for Word Sense Disambiguation. Applied Intelligence 7, 5–26 (1997). https://doi.org/10.1023/A:1008212119713
Issue Date:
DOI: https://doi.org/10.1023/A:1008212119713