Abstract
Exploratory analysis is an area of increasing interest in the computational linguistics arena. Pragmatically speaking, exploratory analysis may be paraphrased as natural language processing by means of analyzing large corpora of text. Concerning the analysis, appropriate means are statistics, on the one hand, and artificial neural networks, on the other hand. As a challenging application area for exploratory analysis of text corpora we may certainly identify text databases, be it information retrieval or information filtering systems. With this paper we present recent findings of exploratory analysis based on both statistical and neural models applied to legal text corpora. Concerning the artificial neural networks, we rely on a model adhering to the unsupervised learning paradigm. This choice appears naturally when taking into account the specific properties of large text corpora where one is faced with the fact that input-output-mappings as required by supervised learning models cannot be provided beforehand to a satisfying extent. This is due to the fact of the highly changing contents of text archives. In a nutshell, artificial neural networks count for their highly robust behavior regarding the parameters for model optimization. In particular, we found statistical classification techniques much more susceptible to minor parameter variations than unsupervised artificial neural networks. In this paper we describe two different lines of research in exploratory analysis. First, we use the classification methods for concept analysis. The general goal is to uncover different meanings of one and the same natural language concept. A task that, obviously, is of specific importance during the creation of thesauri. As a convenient environment to present the results we selected the legal term of “neutrality”, which is a perfect representative of a concept having a number of highly divergent meanings. Second, we describe the classification methods in the setting of document classification. The ultimate goal in such an application is to uncover semantic similarities of various text documents in order to increase the efficiency of an information retrieval system. In this sense, document classification has its fixed position in information retrieval research from the very beginning. Nowadays renewed massive interest in document classification may be witnessed due to the appearance of large-scale digital libraries.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agirre, E. & Rigau, G. (1996). Word sense disambiguation using conceptual density. In Int. Conference on Computational Linguistics.
Allen, J. F. (1987). Natural Language Understanding. Benjamin/Cummings, Menlo Park.
Armstrong, S. (ed.) (1994). Using Large Corpora. MIT Press, Cambridge, MA.
Ashley, K. D. (1990). Modeling Legal Argument. MIT Press, Cambridge, MA.
Bayer, T., Renz, I., Stein, M. & Kressel, U. (1996). Domain and language independent feature extraction for statistical text categorization. In Workshop on Language Engineering for Document Analysis and Recognition.
Belew, R.K. (1987). A connectionist approach to conceptual information retrieval. In Int. Conference on Artificial Intelligence and Law.
Belkin, N. J. & Croft, W. B. (1992). Information filtering and information retrieval: Two sides of the same coin? Communications of the ACM, 35(12).
Bench-Capon, T. (1993). Neural networks and open texture. In Int. Conference on Artificial Intelligence and Law.
Bing, J. (1987). Designing text retrieval systems for conceptual searching. In Int. Conference on Artificial Intelligence and Law.
Bishop, C. M. (1995). Neural Networks for Pattern Recognition. Oxford University Press, Oxford.
Blackmore, J. & Miikkulainen, R. (1995). Visualizing high-dimensional structure with the incremental grid growing network. In Int. Conference on Machine Learning.
Bruce, R. & Wiebe, J. (1994). Word-sense disambiguation using decomposable models. In Annual Meeting of the Association for Computational Linguistics.
Carpenter, G. A. & Grossberg, S. (1988). The ART of adaptive pattern recognition by a self-organizing neural network. Computer 21(3).
Charniak, E. (1993). Statistical Language Learning. MIT Press, Cambridge, Mass.
Church, K. W. & Mercer, R. L. (1993). Introduction to the special issue on computational linguistics using large corpora. Computational Linguistics, 19(1).
Church, K. W. & Rau, L. F. (1995). Commercial applications of natural language processing. Communications of the ACM, 38(11).
Cottrell, M. & De Bodt, E. (1996). A Kohonen map representation to avoid misleading interpretations. In European Symposium on Artificial Neural Networks.
Cross, G. R. & De Bessonet, C. G. (1985). Representation of legal knowledge for conceptual retrieval. Information Processing & Management 21(1).
Cunningham, H., Wilks, Y. & Gaizauskas, R. J. (1996). New methods, current trends and software infrastructure for NLP. In Int. Conference on New Methods in Language Processing.
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K. & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41.
Dick, J. P. (1991). Representation of legal text for conceptual retrieval. In Int. Conference on Artificial Intelligence and Law.
Evans, D. A. & Zhai, C. (1996). Noun-phrase analysis in unrestricted text for information retrieval. In Annual Meeting of the Association for Computational Linguistics.
Fritzke, B. (1994). Growing cell structures: A self-organizing network for unsupervised and supervised learning. Neural Networks 7(9).
Fritzke, B. (1995). Growing grid: A self-organizing network with constant neighborhood range and adaptation strength. Neural Processing Letters 2(5).
Fritzke, B. (1996). Growing self-organizing networks: Why? In European Symposium on Artificial Neural Networks.
Gelbart, D. & Smith, J. C. (1993). FLEXICON: An evaluation of a statistical ranking model adapted to intelligent legal text management. In Int. Conference on Artificial Intelligence and Law.
Greenleaf, G., Mowbray, A. & van Dijk, P. (1995). Representing and using legal knowledge in integrated decision support systems: DataLex WorkStations. Artificial Intelligence & Law, 3(1–2).
Grefenstette, G. (1994). Explorations in Automatic Thesaurus Discovery. Kluwer, Boston.
Groendijk, C. & Oskamp, A. (1993). Case recognition and strategy classification. In Int. Conference on Artificial Intelligence and Law.
Groendijk, C. (1992). Neural schemata in automated judicial problem solving. In Legal Knowledge-Based Systems — Information Technology and Law (JURIX'92).
Hafner, C. D. (1981). An Information Retrieval System Based on a Computer Model of Legal Knowledge. UMI Research Press, Ann Arbor.
Hearst, M. A. (1994). Multi-paragraph segmentation of expository discourse. In Annual Meeting of the Association for Computational Linguistics.
Hobbs, J. R., Appelt, D. E., Bear, J. & Israel, D. (1994). FASTUS: A system for extracting information from text. In ARPA Human Language Technology Workshop.
Höfferer, M., Knaus, B. & Winiwarter, W. (1994). Adaptive information extraction from online messages. In RIAO Conference, Intelligent Multimedia Information Systems and Management.
Honkela, T. Pulkki, V. & Kohonen, T. (1995) Contextual relations of words in Grimm tales analyzed by self-organizing maps. In Int. Conference on Artificial Neural Networks.
Jacobs, P. S. & Rau, L. F. (1990) SCISOR: Extracting information from on-line news. Communications of the ACM 33(11).
Jain, A. K. & Dubes, R. D. (1988). Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs.
Jain, A. K. & Mao, J. (1996). Artificial neural networks: A tutorial. Computer 29(3).
Jaquemin, C. (1994). FASTR: A unification-based front-end to automatic indexing. In RIAO Conference, Intelligent Multimedia Information Systems and Management.
Jing, Y. & Croft, W. B. (1994). An association thesaurus for information retrieval. In RIAO Conference, Intelligent Multimedia Information Systems and Management.
Kangas, J., Kohonen, T. & Laaksonen, J. (1990). Variants of self-organizing maps. IEEE Transactions on Neural Networks 1.
Karov, Y. & Edelman, S. (1996). Learning similarity-based word sense disambiguation from sparse data. Technical Report CS-TR96–05, Department of Applied Mathematics and Computer Science, The Weizmann Institute of Science.
Kaski, S. & Lagus, K. (1996) Comparing self-organizing maps. In Int. Conference on Artificial Neural Networks.
Keane, S., Ratnaike, V. & Wilkinson, R. (1996). Hierarchical news filtering. In Int. Conference on Practical Aspects of Knowledge Management.
Köhle, M. & Merkl, D. (1996). Visualizing similarities in high dimensional input spaces with a growing and splitting neural network. In Int. Conference on Artificial Neural Networks.
Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics 43.
Kohonen, T. (1988). Introduction to neural computing. Neural Networks 1.
Kohonen, T. (1989). Self-Organization and Associative Memory. Springer-Verlag, Berlin.
Kohonen, T. (1990). The self-organizing map. Proceedings of the IEEE 78.
Kohonen, T. (1993). Generalizations of the self-organizing map. In Int. Joint Conference on Neural Networks, Nagoya. Japan.
Kohonen, T. (1995). Self-Organizing Maps. Springer-Verlag, Berlin.
Konstantinou, V., Sykes, J. & Yannopoulos, G. N. (1993). Can legal knowledge be derived from legal texts? In Int. Conference on Artificial Intelligence and Law.
Kozima, H. & Furugori, T. (1993). Similarity between words computed by spreading activation on an English dictionary. In Conference of the European Chapter of the ACL.
Kraaijveld, M. A., Mao, J. & Jain, A. K. (1992). A non-linear projection method based on Kohonen's topology preserving maps. In Int. Conference on Pattern Recognition.
Lagus, K., Honkela, T., Kaski, S. & Kohonen, T. (1996). Self-organizing maps of document collections: A new approach to interactive exploration. In Int. Conference on Knowledge Discovery and Data Mining.
Lewis, D. D. & Sparck Jones, K. (1996). Natural language processing for information retrieval. Communications of the ACM 39.
Lin, X., Soergel, D. & Marchionini, G. (1991). A self-organizing semantic map for information retrieval. In Int. ACM SIGIR Conference on Research and Development in Information Retrieval.
Maarek, Y. S. & Smadja, F. Z. (1989). Full text indexing based on lexical relations — an application: Software libraries. In ACM SIGIR Int. Conference on Research and Development in Information Retrieval.
Maes, P. (1994). Agents that reduce work and information overload. Communications of the ACM 37(7).
Martinetz, T., Berkovich, S. G. & Schulten, K. J. (1993). Neural-gas network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks 4(4).
McCulloch, W. S. & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5.
Merkl, D. & Rauber, A. (1997) Alternative ways for cluster visualization in self-organizing maps. In Workshop on Self-Organizing Maps.
Merkl, D. & Rauber, A. (1997). On the similarity of eagles, hawks, and cows: Visualization of similarity in self-organizing maps. In Int. Workshop Fuzzy-Neuro-Systems.
Merkl, D., Tjoa, A. M. & Vieweg, S. (1992). BRANT: An approach to knowledge based document classification in the information retrieval domain. In Int. Conference on Database and Expert Systems Applications.
Merkl, D., Schweighofer, E. & Winiwarter, W. (1994). CONCAT: Connotation analysis of thesauri based on the interpretation of context meaning. In Int. Conference on Database and Expert Systems Applications.
Merkl, D. (1995). A connectionist view on document classification. In Australasian Database Conference.
Merkl, D. (1995). Content-based document classification with highly compressed input data. In Int. Conference on Artificial Neural Networks.
Merkl, D. (1995). The effects of lateral inhibition on learning speed and precision of a self-organizing feature map. In Australian Conference on Neural Networks.
Merkl, D. (1997). Exploration of document collections with self-organizing maps: A novel approach to similarity representation. In European Symposium on Principles of Data Mining and Knowledge Discovery.
Miikkulainen, R. (1991). Self-organizing process based on lateral inhibition and synaptic resource redistribution. In Int. Conference on Artificial Neural Networks.
Minsky, M. L. & Papert, S. (1969). Perceptrons: An introduction to computational geometry. MIT-Press, Cambridge, MA.
Mital, V., Stylianou, Y. & Johnson, L. (1991) Conceptual information retrieval in litigation support systems. In Int. Conference on Artificial Intelligence and Law.
De Mulder, R. V. & van Noortwijk, C. (1994). A system for ranking documents according to their relevance to a (legal) concept. In RIAO Conference, Intelligent Multimedia Information Systems and Management.
Ng, H. T. & Lee, H. B. (1996). Integrating multiple knowledge sources to disambiguate word sense; an exemplar based approach. In Annual Meeting of the Association for Computational Linguistics.
Nomoto, T. & Nitta, Y. (1994). A grammatico-statistical approach to discourse partitioning. In Int. Conference on Computational Linguistics.
O'Riordan, A. and Sorensen, H. (1995). An intelligent agent for high-precision text filtering. In Int. Conference on Information and Knowledge Management.
Paice, C. D. (1991). A thesaural model of information retrieval. Information Processing & Management 27(5).
Panyr, J. (1987). Vektorraum-Modell und Clusteranalyse in Information-Retrieval-Systemen (Vector-space model and cluster analysis in information retrieval systems). Nachrichten für Dokumentation 38.
Pao, Y.-H. (1989). Adaptive Pattern Recognition and Neural Networks. Addison-Wesley, Reading, MA.
Pereira, F. C. N., Tishby, N. and Lee, L. (1993). Distributional clustering of English words. In Annual Meeting of the Association for Computational Linguistics.
Philipps, L. (1989). Are legal decisions based on the application of rules or prototype recognition? Legal science on the way to neural network. In Pre-Proceedings of the 3rd Int. Conference on Logica, Informatica, Diritto.
Riloff, E. and Lehnert, W. (1994). Information extraction as a basis for high-precision text classification. ACM Transactions on Information Systems 12(3).
Ripley, B. D. (1996). Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge.
Ritter, H. & Kohonen, T. (1989). Self-organizing semantic maps. Biological Cybernetics 61.
Rose, D. E. & Belew, R. K. (1989). Legal information retrieval: A hybrid approach. In Int. Conference on Artificial Intelligence and Law.
Rose, D. E. (1994). A Symbolic and Connectionist Approach to Legal Information Retrieval. Lawrence Erlbaum, Hillsdale.
Rumelhart, D. E. & Zipser, D. (1986). Feature discovery be competitive learning. In Rumelhart, D. E. & McClelland, J. L. (eds.), Parallel Distributed Processing: Explorations in the microstructure of cognition. MIT-Press, Cambridge, MA.
Rumelhart, D. E., Hinton, G. E. & Williams, R. J. (1986). Learning internal representations by error propagation. In Rumelhart, D. E. & McClelland, J. L. (eds.), Parallel Distributed Processing: Explorations in the microstructure of cognition. MIT-Press, Cambridge, MA.
Salton, G. & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management 24.
Salton, G. & McGill, M. J. (1983). Introduction to Modern Information Retrieval. McGraw-Hill, New York.
Salton, G. (1989). Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, Reading.
Schütze, H. & Pedersen, J. O. (1994). A cooccurrence-based thesaurus and two applications to information retrieval. In RIAO Conference, Intelligent Multimedia Information Systems and Management.
Schütze, H., Pedersen, J. O. & Hearst, M. A. (1994). Xerox TREC 3 report: Combining exact and fuzzy predictors. In Text Retrieval Conference.
Schweighofer, E. & Scheithauer, D. (1996). The automatic generation of hypertext links in legal documents. In Int. Conference on Database and Expert Systems Applications.
Schweighofer, E. & Winiwarter, W. (1993a). Legal expert system KONTERM — automatic representation of document structure and contents. In Int. Conference on Database and Expert Systems Applications.
Schweighofer, E. & Winiwarter, W. (1993b). Refining the selectivity of thesauri by means of statistical analysis. In Third Int. Congress on Terminology and Knowledge Engineering.
Schweighofer, E., Winiwarter, W. & Merkl, D. (1995). Information filtering: The computation of similarities in large corpora of legal texts. In Int. Conference on Artificial Intelligence and Law.
Schweighofer, E. (1995). Wissensrepräsentation in Information Retrieval-Systemen am Beispiel des EU-Rechts (Knowledge Representation in Information Retrieval Systems, An Investigation Based on the Example of EU Law). PhD thesis, University of Vienna.
Sergot, M. J., Sadri, F., Kowalski, R. A., Kriwaczek, F., Hammond, P. & Cory, H. T. (1986). The British Nationality Act as a logic program. Communications of the ACM 29(5).
Sharkey, N. (ed.) (1992). Connectionist Natural Language Processing. Intellect, Norwood.
Simitis, S. (1970). Informationskrise des Rechts und Datenverarbeitung (Information Crisis in Law and Data Processing). Müller, Karlsruhe.
Sirosh, J. & Miikkulainen, R. (1993). How lateral interaction develops in a self-organizing feature map. In IEEE Int. Conference on Neural Networks.
Sirosh, J. & Miikkulainen, R. (1994). Cooperative self-organization of afferent and lateral connections in cortical maps. Biological Cybernetics 71.
Smeaton, A. F., O'Donnell, R. & Kelledy, F. (1994). Indexing structures derived from syntax in TREC-3: System description. In Text Retrieval Conference.
Smith, J. C., Gelbart, D., MacCrimmon, K., Atherton, B., McClean, J., Shinehoft, M. & Quintana, L. (1995). Artificial intelligence and legal discourse: The Flexlaw legal text management system. Artificial Intelligence & Law 3(1–2).
Strzalkowski, T., Carballo, J. P. & Marinescu, M. (1994). Natural language information retrieval: TREC-3 report. In Text Retrieval Conference.
Susskind, R. E. (1987). Expert Systems in Law. Oxford University Press, Oxford.
Tapper, C. (1982). An Experiment in Use of Citation Vectors in the Area of Legal Data, Complex 9/82. Universitetsfortlaget, Oslo.
Tokunaga, T., Iwayama, M. & Tanaka, H. (1995). Automatic thesaurus construction based on grammatical relations. In Int. Joint Conference on Artificial Intelligence.
Turtle, H. R. & Croft, W. B. (1991). Evaluation of an inference network-based retrieval model. ACM Transactions on Information Systems 9(3).
Turtle, H. R. & Croft, W. B. (1992). A comparison of text retrieval models. Computer Journal 35.
Turtle, H. R. (1995). Text retrieval in the legal world. Artificial Intelligence & Law 3(1–2).
Ultsch, A. (1993). Self-organizing neural networks for visualization and classification. In Opitz, O., Lausen, B. & Klar, R. (eds.), Information and Classification — Concepts, Methods, and Applications. Springer-Verlag, Berlin.
van Opdorp, G. J., Walker, R. F., Schrickx, C., Groendijk, C. & van den Berg, P. H. (1991). Networks at work: A connectionist approach to non-deductive legal reasoning. In Int. Conference on Artificial Intelligence and Law.
Wan, W. & Fraser, C. (1994). Multiple Kohonen self-organizing maps: Supervised and unsupervised formation with application to remotely sensed image analysis. In Australian Conference on Neural Networks.
Werbos, P. J. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioural Sciences. PhD thesis, Harvard University.
Werbos, P. J. (1994) The Roots of Backpropagation — From Ordered Derivatives to Neural Networks and Political Forecasting. John Wiley & Sons, New York.
Wermter, S. (1995). Hybrid Connectionist Natural Language Processing. Chapman and Hall, London.
Winiwarter, W. (1995). MIDAS — the morphological component of the IDA system for efficient natural language interface design. In Int. Conference on Database and Expert Systems Applications.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Merkl, D., Schweighoffer, E. & Winiwarter, W. Exploratory analysis of concept and document spaces with connectionist networks. Artificial Intelligence and Law 7, 185–209 (1999). https://doi.org/10.1023/A:1008365524782
Issue Date:
DOI: https://doi.org/10.1023/A:1008365524782