1 Wordnets

Since its inception a quarter century ago, Princeton WordNet [PWN] (Miller 1995; Fellbaum 1998) has had a profound influence on research and applications in lexical semantics, computational linguistics and natural language processing. The numerous uses of this lexical resource have motivated the building of wordnetsFootnote 1 in several dozen languages, including even a “dead” language, Latin. This special issue looks at certain aspects of wordnet construction and organisation.

There are diverse methods of wordnet construction, relying on different underlying theories.Footnote 2 The most common method is to translate PWN, using corpora and bilingual or multilingual dictionaries to assist in transferring the structure from the source to the target wordnet. Translation, however, does not do justice to the many lexical, semantic and typological cross-lingual differences. Non-trivial customization is required, and it is a significant challenge. Some wordnet construction projects have relied to a varying degree on extraction of lexico-semantic relations from large corpora to fill in translation gaps (Hamp and Feldweg 1997; Nadig et al. 2008; Montazery and Faili 2010; Vintar and Fišer 2011), while others have bootstrapped larger coverage from a manually encoded core typically based on traditional dictionaries or corpora (Chugur et al. 2001; Prószéky and Miháltz 2002; Bilgin et al. 2004; Lindén et al. 2012). Still others have abandoned translation altogether in favour of creating a wordnet from scratch, aiming to reflect accurately the specific linguistic and cultural properties of the encoded concepts.Footnote 3

On the other hand, the growing recognition of the need for cross-language connections among wordnets—to support multi-lingual research and applications—has led to the practice of ensuring that new wordnets are mappable to and from PWN,Footnote 4 in order to enhance their interoperability. One should note that PWN is a bit of a moving target: it too undergoes constant development, and its new versions appear regularly.

2 Relations

The backbone of any wordnet is a network of instances of a fixed set of relations, which hold among words and sets of roughly synonymous words known as synsets. Synsets are interconnected by pointers inspired by lexico-semantic relations in lexicography. Those relations, however, link lexical units,Footnote 5 while a synset in PWN and in many other wordnets represents a lexicalised concept. Although the labels may be the same (hyponymy, hypernymy, meronymy, and so on), it is not straightforward to transfer a linguistic definition of a semantic relation from word pairs to concept or synset pairs. Moreover, almost all wordnets encode relations both between lexical units and between synsets. Antonymy, a lexical relation, is an example of the former; hypernymy, a conceptual relation, exemplifies the latter.

One of the purposes of a wordnet is to describe lexical meaning. How much should the two types of relations contribute to this goal, and how do they differ in principle? We know, for example, that antonymy signals a clear semantic opposition, while many derivational relations (white → whiteness, slow → slowly, knight → knighthood, …) often capture a meaning-preserving formal operation.

Cross-lingual connections among wordnets could benefit from a set of common relations across wordnets, but that is easier said than done. Designers sometimes unquestioningly adopt the repertory from PWN and then perhaps customise it. The PWN set can certainly be considered as core: it figures in most wordnets.Footnote 6 Still, how (if at all) such relations are defined makes for non-trivial differences among wordnets. Hyponymy and hypernymy are consistent for nouns, but not so much for verbs and adjectives. Troponymy puts verbs into a hierarchy in PWN, but it is hyponymy in GermaNet (Kunze 1999), EuroWordNet(Vossen 1998) and plWordNet (Piasecki et al. 2009).

There is also variety among verb entailment relations in wordnets. PWN (from version 1.5 onwards) features entailment in general, encompassing presupposition and proper inclusion, though the different types of entailment have been discussed in the WordNet literature. On the other hand, GermaNet (Kunze 1999) and EuroWordNet (Vossen 1998, p. 94) actually encode three types: subevent (“part of” entailment), causal or causation (resultative) relation, and entailment proper (backward presupposition).

Meronymy and its inverse, holonymy, are usually divided into several subtypes, but the subtype lists differ among wordnets. Many other relations, sometimes specific to one wordnet, have been suggested. DanNet (Pedersen et al. 2009) proposes functional relations, such as for example used_for modelled on the telic qualia role of the Generative Lexicon (Pustejovsky 1998). The question arises, then: what kind of relations can be and should be present in wordnets? What criteria should govern the selection of a set of relations for a new wordnet? Relations are what distinguishes not only wordnets, but also languages which such wordnets are meant to model. A deeper, more systematic look appears necessary. It is that need that our special issue has sought to meet.

A popular view is that a wordnet consists of interrelated synsets, yet this is only part of the story. Instances of lexical relations—a substantive part of the overall network—add a lot to the descriptive power of many wordnets. There also are relations of a meta-descriptive character, such as the domain relation in PWN. It is, therefore, important to determine the lowest-level building block of a wordnet. It cannot be both a synset and a lexical unit. This becomes even more important when we consider how minimal the synset definition is: a set of (near-)synonyms which identify a lexicalised concept represented by the shared meaning of synset members. Meaning and synonymy, subject to intensive research in linguistics, have been largely left to the intuition of people who build wordnets. Intuitions vary, so it is better if the nature of synsets is precise, allowing such people to maintain consistency. Synsets play a central role in a complete wordnet, and that is why the stability of editing decisions during its construction is a major concern.

3 The papers in the special issue

Semantic relations are central to the theory and practice of wordnet construction and applications. There is a lot of literature of the subject of wordnets and relations, but no systematic overview. This special issue is an attempt to bring together recent work in this area. The call for papers asked for contributions on any of a long list of relevant topics:

  • lexico-semantic relations in linguistics and in wordnets,

  • wordnets versus other types of thesauri, and relations therein,

  • the lexicographic theories and practices versus wordnet-creation practices,

  • mono-lingual and multi-lingual considerations in the creation of a wordnet,

  • the issues around translating a wordnet into another language,

  • comparing wordnets for one language and between languages from the standpoint of relation sets,

  • automatic extraction of lexical semantic relations and the role of large corpora in practical wordnet development,

  • evaluation of lexico-semantic relations in wordnets—consistency, coverage, relevance for applications.

Sections 1 and 2 discussed those topics which are accounted for, to a varying degree, in the six papers ultimately accepted for publication in this special issue. This section presents our perspective on the papers, and attempts to fit them into the broader picture implied by the foregoing list of topics. Section 4 will briefly examine a few themes not covered in this issue. It may inspire further discussion on wordnets and relations.

“The chicken-and-egg problem in wordnet design: synonymy, synsets and constitutive relations” by Maziarz, Piasecki & Szpakowicz

The authors look at wordnet basics: synsets, lexicalised concepts, lexical units, synset relations, conceptual relations, and lexical relations. They observe that one can define a synset, frugally, as “a set of synonyms that serve as identifying definitions of lexicalised concepts” only if the notions of synonymy and lexicalised concepts are well understood. Most theories of synonymy cannot be made operational for use in wordnet development, and lexicalised concepts are not linguistic objects. The authors argue that in most wordnets conceptual relations (synset relations) coexist, somewhat uneasily, with lexico-semantic relations which operate among lexical units. In fact, synsets consists of lexical units but, a little confusingly, the relations at both levels have the same names. Maziarz et al. propose a granularity radically different than that of a network of synsets. The lexical unit becomes the basic building block. Small sets of lexical units are grouped into synsets because they share certain lexico-semantic relations: those lexical units cannot be semantically distinguished with respect to their connectivity in the wordnet.

Not all lexico-semantic relations are helpful in identifying synsets by way of connectivity. A constitutive relation ought to be well-established in linguistics, defined with sufficient specificity and useful in generalising the lexical meaning. A study of the relevant linguistic and wordnet literature suggested a set of constitutive relations, later illustrated in practice by a large-scale implementation: plWordNet, a semi-automatically constructed very large wordnet for Polish. The topological criteria in shaping the wordnet structure have been complemented by references to stylistic register, and to the aspect and semantic class of verbs. Stylistic register is invoked in several definitions of plWordNet’s lexico-semantic relations. A relation can only link lexical units of compatible register values. A register compatibility table is part of the plWordNet model. Likewise, aspect and semantic class constrain lexico-semantic relations for verbs.

The constitutive relations aptly differentiate units with a significant difference of meaning, yet do not require a continual introspection on the near-identity of the meaning of concepts. This form of minimal commitment in wordnet construction allows a practical reduction of the conceptual basis upon which one can consistently build a large wordnet.

“Large, huge or gigantic? Identifying and encoding intensity relations among adjectives in WordNet” by Sheinman, Fellbaum, Julien, Schulam & Tokunaga

The paper focuses on adjectives, a lexical category not common to all languages and ignored or neglected in many wordnets. In PWN, three broad classes of adjectives are distinguished: participles based on verbs, “pertainyms” derived from nouns and core, mostly descriptive, adjectives. The latter constitute the bulk of the lexicon but, because they are not derived, they cannot straightforwardly point to other words and synsets in PWN. Their semantic representation is based on the observation that members of antonymous adjective pairs like hot-cold and young-old are strongly associated with each other in speakers’ minds and also show strong textual co-occurrence preferences.

PWN currently represents core adjectives in terms of semantic clusters consisting of two antonymous adjectives, each ringed by a set of “similar” adjectives and adjective synsets. Sheinman et al. note that this representation not only is vague but also misses an essential meaning component of many descriptive adjectives: scalarity. They propose to introduce scales to represent an attribute, such as size or weight, shared by all adjectives which occupy a point on that scale. The relative order of the adjectives can be derived by mining corpus data using lexical-semantic patterns which reveal the asymmetry of intensity between pairs of adjectives. Arcs from adjectives in the current clusters to specific points on the scales can be seen as new relations. Moreover, scale members such as large, huge and gigantic share a link to the noun synset which expresses the attribute signalled by the scale, for example size. Sheinman et al. make a case for a richer semantic representation of the most frequent English adjectives and for the concomitant benefits to a wide range of applications.

“Evocation: analyzing and propagating a semantic link based on free word association” by Ma

The author discusses another new relation for wordnets, but the point of view is different than in the preceding papers. Unlike in Maziarz et al., the evocation relation (Boyd-Graber et al. 2006) is psychologically motivated. In contrast to Sheinman et al., evocation is not limited to one part of speech. The objective is to improve the connectivity among synsets, including those in different parts of speech. Evocation has been defined as “a quantitative measure of the association between directed sense pairs based on human judgment”—how much a concept lexicalised by one or more words brings to mind another concept.

Evocation is similar to conceptual wordnet relations in that it associates word senses. Often, however, it links words across parts of speech and concepts of different kinds, for example entities and events. The lack of such meaningful associations is presented as a limitation of wordnets. For example, Ma cites an experiment which shows that only 38 % of words linked by evocation are connected in PWN by paths of wordnet relations no longer than ten edges.

It is hard to make judgment on evocation consistent and replicable. That is why Ma’s main goal has been to build “a reliable and extendable dataset of concepts evoked by, and evoking, other concepts” on the basis of “disambiguated free word association data”. Those data are responses collected from experiment participants after they have been shown a word.

Word associations are semantically ambiguous. Ma proposes a novel methodology of transforming word associations into pairs of sense-specific words. A word-sense disambiguation algorithm assigns senses to stimulus-response word pairs. The assignments are then manually verified. Ma also studies manual evocation strength assignment and propagation of the evocation links by crowd-sourcing. Unlike synset relations and lexical relations, the strength of evocation can be measured. That can be useful in applications, for example to define context for word-sense disambiguation more adequately.

“Using part-whole relations for automatic deduction of compound-internal relations in GermaNet” by Henrich, Hinrichs & Barkey

There are four types of meronymy in GermaNet: component, member, substance and portion. Footnote 7 The authors explain this subdivision of the general part-of relation, and show how the more specific relations can help leverage a certain form of automated expansion of GermaNet. The very productive system of compound-formation in German also introduces semantic relations inside a compound: can they be deduced?

Deduction rules are based on the syntactic and semantic properties of German compounds. The rules mark compound-internal relations which describe semantic associations among the compound’s constituents. The semantic properties considered in this work are those signalled by the existing meronymy: how a part (a constituent) is related to the whole (the compound). The authors want “to provide a more in-depth analysis for the set of nominal compounds that exhibit” part-whole relations—of the kind very well described in GermaNet.

Given a part-whole connection, the holonymy can be the whole compound, the modifier or the head, depending on the relation subtype and the compound structure. Henrich et al. identify five classes of dependencies between a compound structure and a part-whole subtype. The classes drive the construction of the deduction rules; for example, a rule can be based on the fact that the compound has its head as a meronym. In each class, different meronymy subtypes promote different semantic relations, for example unit measure, member or spatio-temporal.

Henrich et al. compare the ratio of meronymy subtypes in GermaNet and PWN. That helps analyse the coverage of meronymy and the generative power of the deduction rules, which have been manually evaluated as achieving the precision of ≈95 %. The strength of the rules lends indirect support to the subdivision of meronymy. Thus the paper neatly illustrates the far-reaching consequences of wordnet design decisions—here, meronymy subtype definitions. The authors explore the connection between the lexical system as it is described in a wordnet and the semantic structure of natural language expressions. Wordnet relations are presented as an element of a broader system.

The scale and domain of this research have been intentionality limited in order to allow the results to be thorough. The work explores interesting links between knowledge described in a large wordnet and automated analysis of the semantic structure of compounds. It can be expanded to the semantic analysis of multi-word expressions. This is a valuable case study in using a wordnet as a knowledge source for semantic parsing. It also serves as a linguistic tool in the study of the language semantic system.

“Tailoring the automated construction of large-scale taxonomies using the Web” by Kozareva and Hovy

The paper tackles an important problem of the fully automated construction of a wordnet only from corpus data, without the help of any kind of knowledge base or structured documents. Automatic wordnet construction could solve several problems. First, wordnets tend to have inevitably limited coverage in terms of vocabulary and word senses. The limitation can be reduced if one relies on massive collections of contemporary language data. Second, “sub-wordnets” for specific domains are hard to build, and lexical meaning in some subdomains changes dynamically, demanding regular updates. Third, wordnet construction is potentially too subjective; even careful manual construction leaves simple errors or erroneous gaps in the network. The list goes on…

Automated methods can help if their coverage and—especially—accuracy are at an appropriately high level. Most existing methods focus on hypernymy, although there are several equally important relations in wordnets. Kozareva and Hovy present a pattern-based, lightly supervised method of constructing a taxonomy by harvesting large amounts of online domain-specific or general text. Such a taxonomy can contain instances of meronymy, so it already is an improvement on the customary methods which usually do not consider this relation. The authors introduce a novel idea of doubly-anchored patterns. There are two anchor seed positions (occupied by words in a known relation) and a slot for a new word. The construction process is minimally supervised by a pre-specified “root concept” (a word representing such a concept) and a low-level concept. The algorithm, however, requires no initial taxonomy. Patterns are used recursively—extracted words shift into seed positions. After several phases of harvesting upwards and downwards (for more generality and more specificity), there follows a taxonomy induction stage based on additional patterns and transformations of the extracted relation graph.

Kozareva and Hovy report very good results for relatively simple domains of animals, people, vehicles and plants. The method illustrates well how wordnet development can benefit from automated methods applied in appropriately selected subdomains. A few open questions remain. For example, it would be interesting to know how far can one go with automated methods in relation to the core wordnet so as to identify the most general, abstract terms. Certain preconditions should also hold for the corpus, words and their distributions if automated methods are to produce a resource accurate enough to rival a manually constructed wordnet. Footnote 8

“On the evaluation and improvement of Arabic WordNet coverage and usability” by Abouenour, Bouzoubaa & Rosso

The paper presents the development of an extended Arabic WordNet [AWN]. Abouenour et al. take a clear perspective of a wordnet as a lexico-semantic resource for language processing. In order to identify the requirements, they first evaluate the unimproved resource’s coverage and usability. The former compares AWN’s size with other wordnets and against the vocabulary extracted from a large corpus. Not only are nouns and verbs taken into account, but also named entities (probably narrowed down to proper names) and broken plurals, specific to Arabic. Proper names, described as a dynamic class, are not part of any core lexicon, but their description can be very useful in applications. (Note how treating a wordnet as a practical language resource influences design decisions.)

Interestingly, AWN’s usability has been defined in terms of the gain from using AWN in Arabic question-answering. Improvement in the quality of passage ranking has been chosen as a measure: how using a wordnet helps promote (in the system’s answer ranking) those documents which include the answer to the user question.

Semi-automation is based on the machine-translated content of several available resources and on the application of lexico-syntactic patterns. Proper names have been added to AWN via automated translation of proper names in YAGO (Suchanek et al. 2007) followed by manual verification. The verb portion of AWN has been expanded by translating the current content of VerbNet (Schuler 2005) and disambiguating the resulting English-Arabic links; a few heuristic rules were applied to the graph of potential links. The coverage of noun synsets has been improved by attaching new hyponyms extracted using lexico-syntactic hyponymy patterns. The average precision of the extracted hyponymy instances was 30–50 %, and fewer than 1,000 correct hyponyms have been added. Footnote 9 In the end, most of the new synsets were proper names.

This method of semi-automatic expansion fits the main evaluation criterion: usability as an improvement in applications. It would be interesting to study the expanded AWN from a linguistic or psycholinguistic point of view. Evaluation based on an application is objective and measurable, but it favours few factors at the expense of many others. It remains an open question what composite picture of the whole lexical system the expanded AWN presents.

4 The lacunae

Even with luck, one special issue cannot accommodate the variety of concerns relevant to wordnets and relations. The call for papers cast a wide net, and the community responded. Winnowing the contributions has left several worthy topics unrepresented. This section is intended as an invitation to an ongoing discussion.

Maziarz et al. propose a linguistically motivated model of a wordnet and implicitly ask: what is a wordnet? Perhaps such a resource can be treated as an embodiment of the relational paradigm of lexical semantics; the authors confront their point of view with definitions adopted in several well-known wordnets, beginning with PWN. Ma advocates a psychologically motivated expansion of a wordnet with relations based on word associations. Kozareva and Hovy vote implicitly for a wordnet as a kind of linguistically rich ontology with a taxonomic structure as the backbone. Abouenour et al. emphasise the role of a wordnet as a language resource in applications. None of those papers, however, poses and answers openly the question of the nature of wordnets. An explicit confrontation of the different points of view and a comparison of wordnets to other types of thesauri is missing. Footnote 10 When does a wordnet with an ever larger and more complicated model stop being a wordnet? Can defining criteria for wordnets be formulated? Are such criteria needed? What kind of perspective on a wordnet—a semantic network, a lexicon, a language-processing resource, an ontology—works better, in what circumstances and for what purpose?

Kozareva and Hovy present an automated construction of part of a wordnet, based on the largest corpus imaginable: the Web. Subgraphs of hypernyms and meronyms are analysed. The method relies on several interesting, innovative ideas, and performs very well in selected concrete domains, but there is no deepened analysis of this method (and others with similar performance proposed in the literature) as a viable tool for wordnet construction. Abouenour et al. present expansion statistics which their semi-automated method affords, but do not say how helpful that method can be for other wordnet developers. No matter what, it seems clear that no fully automated method can expand a wordnet entirely credibly, and fully automated construction from scratch is downright impossible. Wide-ranging manual control and verification is indispensable. What, then, is the gain and what is the clear benefit of automated methods? Do they really decrease the workload and the cost of wordnet development? Do they improve the quality? These questions must wait for in-depth studies.

It is easy to ask about quality, but the quality of a wordnet requires operational definitions and effective evaluation methods which are currently lacking. The large size of most language resources makes intrinsic, manual evaluation prohibitively costly. Wordnets are a case in point. Their structure depends non-trivially on the wordnet editors’ subjective decisions, and they feature complicated sets of relations. Extrinsic evaluation is no less troublesome, given that wordnets are meant for a very wide range of potential applications.

The evaluation of wordnets has been barely touched upon in this special issue. Abouenour et al. report on an evaluation via an application of AWN in question-answering. While such an evaluation is commendably objective, the most impressive improvement concerns proper names, a rather atypical category. A comprehensive evaluation of a wordnet must take into account multiple factors. The funding for wordnet construction is generally so scant that such evaluation is not feasible. A haphazard but convincing evaluation can only be carried out by a large community of wordnet users: the proof of the pudding is in the eating. We hope to see the problem of wordnet evaluation addressed comprehensively in future research.