Advertisement

Dualism in Topical Relevance

Conference paper
  • 3k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12036)

Abstract

There are several concepts whose interpretation and meaning is defined through their binary opposition with other opposite concepts. To this end, in this paper we elaborate on the idea of leveraging the available antonyms of the original query terms for eventually producing an answer which provides a better overview of the related conceptual and information space. Specifically, we sketch a method in which antonyms are used for producing dual queries, which can in turn be exploited for defining a multi-dimensional topical relevance based on the antonyms. We motivate this direction by providing examples and by conducting a preliminary evaluation that shows its importance to specific users.

Keywords

Information retrieval Exploratory search Dualism Antonyms 

1 Introduction: Why Dualism Is Important

Dualism denotes the state of two parts. The term was originally coined to denote co-eternal binary opposition and has been especially studied in philosophy. For example, there is duality in ethics (good - bad), in human beings (man - Nietzsche’s übermensch or man - god) and in logic (true - false). In addition, dualism determines in a great extent our everyday lives (ugly - beautiful, happy - unhappy, etc.), and our relations with other people (rich - poor, black - white, love - hate, etc.). None of these concepts can be understood without their dual concepts, since this duality and opposition generates their meaning and interpretation. Dualism is also crucial in mathematics and physics (e.g., matter - antimatter), and is the power behind our whole information society and our binary data.

Moving from philosophy, sciences and everyday life to information retrieval, we find a very vague situation. Users of search engines are ‘dictated’ to provide a very concise and specific query that is extremely efficient for focalized search (e.g., looking for a specific hotel). On the other hand, studies show that \(60\%\) of user tasks are of exploratory nature [12]. In such tasks users do not accurately know their information need and can not be satisfied by a single ‘hit’ [5]. Consequently, users spend a lot of time reformulating queries and investigating results, in order to construct a conceptual model regarding their information need. Information needs that include non-monosemous terms can be considered such exploratory tasks. However, the simplicity of inserting terms in an empty text box and ‘magically’ return the most relevant object(s), will always be a desired feature.

In this paper we elaborate on the idea of leveraging the available antonyms of the original query terms (if they exist), for eventually producing an answer which provides a better overview of the related information and conceptual space. We sketch a method in which antonyms are used for producing dual queries, which in turn can be exploited for defining a multi-dimensional topical relevance. This approach can be applied on demand, helping users to be aware of the various opposing dimensions and aspects of their topic of interest. A preliminary evaluation shows the value of the approach for some exploratory tasks and users.

To the best of our knowledge, the proposed direction is not covered by the existing literature. Antonyms have been studied in Fuzzy Logic [7] showing a relation with negates. In the IR domain, query expansion methods are based on synonyms and semantically related terms, but do not exploit antonyms explicitly, while in relevance and pseudo-relevance feedback techniques the antonyms are essentially penalized [1]. Results diversification can produce a kind of dual clusters, but this is neither guaranteed nor controlled [3].

2 Motivating Examples

“Capitalism and War”. Consider a user exploring the relationship between capitalism and war. The user submits to a WSE (Web Search Engine) the query “capitalism and war” and starts inspecting the results. The left part of Fig. 1 shows the top-5 results for this query from a popular WSE. The results include articles about the connection of capitalism with war from research and academic domains, as well as from socialistic, communistic and theological sites. Considering a different direction, the user might also be interested about how capitalism can support peace, the dual of war. The top-5 results for the query “capitalism and peace” are shown at the right side of Fig. 1. They contain a wikipedia and a research article about the Capitalist Peace Theory, and articles about the importance of capitalism for the prosperity of modern societies and its association to peace from policy research organizations.

Analogously, since socialism is the economic system that opposes capitalism, the user could be interested about how socialism may promote war or support peace, by inspecting the results of the queries “socialism and war” and “socialism and peace” respectively. The top-5 results for each of the above queries are shown in Fig. 2. The results for the former query include the Socialism and War pamphlet written by Lenin, a collection of articles by the economist and philosopher Friedrich Hayek, a list of articles from two marxist domains, and a critical article for both left and right views from the Foundation for Economic Education. For the latter query, the results include articles connecting socialism with peace, like a chapter from the Encyclopedia of Anti-Revisionism, a wikipedia article about the theoretical magazine Problems of Peace and Socialism, and an article from a site supporting a far left U.S. Party.
Fig. 1.

Top-5 results for “capitalism and war” and “capitalism and peace”

Fig. 2.

Top-5 results for “socialism and war” and “socialism and peace”

The above hits indicate interesting directions to the original information need of the user. We argue that users should get aware of these directions for a better exploration of the domain at hand, since they can provide a more comprehensive view of the information and conceptual space. Furthermore, the exploration of these directions let available supportive or counter arguments of dual concepts to emerge, leading to better informed and responsible humans and citizens.

“Aloe”. A comprehensive view of the various different directions can be beneficial also for reducing false-positive results. For example, consider a pregnant woman that was advised to take aloe vera by mouth to relieve digestive discomfort. To check if this is true, she submits to a WSE the query “aloe vera indications”. However, since aloe can stimulate uterine contractions, increasing the risk of miscarriage or premature birth, it is crucial to know also its contraindications. The proposed direction can alleviate this problem, because this information would be contained in the results of the query “aloe vera contraindications”.

3 The Gordian Knot Solution

One can imagine various ways for leveraging antonyms. We shall hereafter use \(t \divideontimes t'\) to denote that the terms t and \(t'\) are antonyms. Building on the “capitalistic” example of the previous section, according to the online dictionary Wordnet1, socialism \(\divideontimes \) capitalism, and war \(\divideontimes \) peace. Now, we can generate all possible queries, denoted by \(\mathcal {Q}\), where non-monosemous terms of the original query are substituted by their dual ones, as expressed by their antonyms. For example, the query “capitalism and war” will generate three extra queries: “socialism and peace”, “capitalism and peace” and “socialism and war”. Based on \(\mathcal {Q}\) we can now define two vector spaces. In the first case, the space has \(|\mathcal {Q}|\) dimensions, where each query is a dimension of the space. Each document is placed in this space according to its relevenace to each query. In the second case we assume a space with only \(\frac{|\mathcal {Q}|}{2}\) dimensions. Each dimension represents a pair of dual queries, where each query in the pair contains the antonyms of the other. We denote with \({\varvec{q}} \divideontimes {\varvec{q}'}\), that the queries \({\varvec{q}}\) and \({\varvec{q}'}\) are dual. For our running example, the first pair is (“capitalism and war”,“socialism and peace”) and the second one is (“capitalism and peace”,“socialism and war”). Each pair defines an axis, therefore the two pairs define a 2D space against which we can evaluate the “value” of each document. For each axis we can consider policies for composing the relevance scores of each document to each member of a dual query.

Generally, there are various criteria that can be considered for assessing the value of each document or set of documents. Such criteria include the bias of documents to specific queries (e.g., the original user query), the purity to a specific query, the overview factor of a document regarding either a dual query or all queries, and the diversity of the returned set of documents with respect to these queries. In general, we need to define appropriate ranking methods, that will take into account the relevance of the documents to the available queries for different criteria. Therefore, we will explore whether the existing multiple-criteria approaches described in [4, 6, 9, 13] are appropriate for the problem at hand.

Regarding the process of finding the corresponding antonyms, we can use existing dictionaries like WordNet for nouns and adjectives or word-embedding antonym detection approaches like [8, 11]. The case of verbs and adverbs is more complicated since they require a kind of grammatical and language analysis (i.e., exist \(\divideontimes \) not exist, lot \(\divideontimes \) total, a lot \(\divideontimes \) bit, etc). There are three categories of antonyms: (a) gradable, (b) relational and (c) complementary. We have gradable antonyms (e.g., hot \(\divideontimes \) cold) in cases where the definitions of the words lie on a continuous spectrum. We have relational antonyms (e.g., teacher \(\divideontimes \) student) in cases where the two meanings are opposite only within the context of their relationship. The rest are called complementary antonyms (e.g., day \(\divideontimes \) night). In general, the selection of the “right” antonyms raises various questions. In many cases more than one antonyms exist, so one should decide which one(s) to select. Sometimes this can depend on the context, e.g., the antonym of “action” is “apathy”, but in terms of physics or sociology the dual of “action” is “reaction”.

Notice that the proposed approach can be exploited in any context where the aim is to retrieve semantically opposing entities, information, etc. As an example consider the Argument Web [2], where the approach could be used for retrieving contradicting arguments and providing support for each one of them. From a system’s perspective, the approach can be realized in various levels and settings. In the setting of an IR system, it can be implemented by changing accordingly the query processor and the ranking module, while in a meta-search setting, by changing the query rewriting, the query forwarding and the ranking components. It could also be exploited in the query autocompletion layer.

4 Preliminary Evaluation

To start with, we have conducted a preliminary evaluation. We have specified 15 information tasks which are shown in Table 1, that can exploit the proposed approach. The tasks are of exploratory nature and were created using the task refinement steps described in [10]. We have identified the following types of tasks: Explore Domain (ED), Medical Treatment (MT), Explore Product Reviews (EPR) and Person Qualities (PQ). For each task we provide a description of the information need, a representative query and the relevant antonyms, which were manually selected from the list of the respective WordNet antonyms.
Table 1.

Evaluation tasks

Id

Type

Information need

Query

Antonyms

\(Q_1\)

ED

is capitalism connected to war?

capitalism and war

socialism, peace

\(Q_2\)

ED

learn about the greek debt crisis

greek debt crisis

profit

\(Q_3\)

ED

learn about 9/11

the truth about 9/11

lie

\(Q_4\)

ED

learn about black holes

black holes

white

\(Q_5\)

ED

search about free will

free will

enslaved

\(Q_6\)

ED

what is open source?

open source

closed

\(Q_7\)

ED

learn about the syrian civil war

syrian civil war

peace

\(Q_8\)

MT

you are pregnant and you were suggested to drink Aloe Vera

aloe vera indications

contraindications

\(Q_9\)

MT

you were suggested that the drug accutane can help with acne

positive accutane experiences

negative

\(Q_{10}\)

EPR

which is the best smartphone?

best smartphone

worst

\(Q_{11}\)

EPR

android phones are better than iphone

android phones are better than iphone

worse

\(Q_{12}\)

EPR

where should I go for my summer?

best summer vacation

worst, winter

\(Q_{13}\)

EPR

which are the safest cars?

most safe cars

less, unsafe

\(Q_{14}\)

PQ

learn about Obama’s peace nobel prize

Obama nobel peace prize

war

\(Q_{15}\)

PQ

is Greek politician XXX smart?

XXX is smart

stupid

We conducted our experiment over 9 female and 25 male users of various ages. For each task, they were given two lists of results. One contained the results of the query from a popular WSE, and the other one was constructed by interleaving the results of the same WSE for the dual queries of this task (i.e., first the top result of the original query, then the first result of its dual, etc.). The two kinds of lists were given in random order for each task. The users were asked to select the most preferred list and to provide a grade of preference taking values in {1, 2, 3, 4, 5}, where 5 means that the selected list was preferred much more than the other one. In the background, when users prefer the results of the dual approach, we change the sign of the score and make it negative. The users were not aware how the lists were constructed and were not guided in any way by the evaluator.

In Fig. 3 we provide two graphs that describe the results of the evaluation. Figure 3 (a), shows the aggregated scores given by all users to each query, while Fig. 3 (b) shows the aggregated scores given by each participant to all queries. Regarding the first one the results are not the expected ones, although we hypothesize that the users mainly penalized the dual approach because of the ‘irrelevant’ results to the original query in terms of query tokens and not in terms of relevant information. For eleven of the queries there is a strong preference towards the non-dual approach. The EPR type of queries belong to this category, showing that users are probably not interested for reviews with the opposite direction of what they are looking for. This is especially true for \(Q_{12}\), where the dual approach provided results about winter vacations and was the least preferred. For two of the tasks, the approaches are almost incomparable. Both of these tasks belong to the MT group. There are also two queries, \(Q_3\) and \(Q_{15}\), where the dual approach is better, especially in the last one. In their comments for these queries, users mention that the selected (i.e., dual) list “provides a more general picture” and “more relevant and interesting results, although contradicting”. Regarding the second graph we have the interesting result that the proposed approach appeals to specific users. It seems that nine users (26% of the participants) have an exploratory nature and generally prefer the dual approach (six of them strongly), while for four of them the two approaches are incomparable. The rest are better served with the non-dual approach. This is an interesting outcome, and in the future we plan to identify the types of users that prefer the dual approach.
Fig. 3.

Graph (a) shows the aggreggated users’ scores for each query, while graph (b) the aggreggated scores of each user for all queries

5 Conclusion and Future Work

We have motivated with examples why it is worth investigating dualism for non-monosemous terms in the context of exploratory search and we have shown its importance at least for some types of users and tasks. For the future, we plan to define the appropriate antonyms selection algorithms and relevance metrics, implement the proposed functionality in a meta-search setting, and conduct a large scale evaluation with real users over exploratory tasks, to identify in which queries the dual approach is beneficial and to what types of users.

Footnotes

References

  1. 1.
    Azad, H.K., Deepak, A.: Query expansion techniques for information retrieval: a survey. Inf. Process. Manag. 56(5), 1698–1735 (2019)CrossRefGoogle Scholar
  2. 2.
    Bex, F., Lawrence, J., Snaith, M., Reed, C.: Implementing the argument web. Commun. ACM 56(10), 66–73 (2013)CrossRefGoogle Scholar
  3. 3.
    Carpineto, C., D’Amico, M., Romano, G.: Evaluating subtopic retrieval methods: clustering versus diversification of search results. Inf. Process. Manag. 48(2), 358–373 (2012)CrossRefGoogle Scholar
  4. 4.
    da Costa Pereira, C., Dragoni, M., Pasi, G.: Multidimensional relevance: a new aggregation criterion. In: Boughanem, M., Berrut, C., Mothe, J., Soule-Dupuy, C. (eds.) ECIR 2009. LNCS, vol. 5478, pp. 264–275. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-00958-7_25CrossRefGoogle Scholar
  5. 5.
    Crawford, D. (ed.): Supporting Exploratory Search, vol. 49. ACM, New York (2006)Google Scholar
  6. 6.
    da Costa Pereira, C., Dragoni, M., Pasi, G.: Multidimensional relevance: prioritized aggregation in a personalized information retrieval setting. Inf. Process. Manag. 48(2), 340–357 (2012)CrossRefGoogle Scholar
  7. 7.
    De Soto, A.R., Trillas, E.: On antonym and negate in fuzzy logic. Int. J. Intell. Syst. 14(3), 295–303 (1999)CrossRefGoogle Scholar
  8. 8.
    Dou, Z., Wei, W., Wan, X.: Improving word embeddings for antonym detection using thesauri and sentiwordnet. In: Zhang, M., Ng, V., Zhao, D., Li, S., Zan, H. (eds.) NLPCC 2018. LNCS (LNAI), vol. 11109, pp. 67–79. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-99501-4_6CrossRefGoogle Scholar
  9. 9.
    Gabrielli, S., Mizzaro, S.: Negotiating a multidimensional framework for relevance space. In: Draper, S.W., Dunlop, M.D., Ruthven, I., van Rijsbergen, C.J. (eds.) MIRA, Workshops in Computing, BCS (1999)Google Scholar
  10. 10.
    Kules, B., Capra, R.: Creating exploratory tasks for a faceted search interface. In: Proceedings of the 2nd Workshop on Human-Computer Interaction (HCIR 2008) (2008)Google Scholar
  11. 11.
    Ono, M., Miwa, M., Sasaki, Y.: Word embedding-based antonym detection using thesauri and distributional information. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 984–989 (2015)Google Scholar
  12. 12.
    Rose, D.E., Levinson, D.: Understanding user goals in web search. In: Proceedings of the 13th International Conference on World Wide Web, WWW 2004, pp. 13–19. ACM Press (2004)Google Scholar
  13. 13.
    Saracevic, T.: Relevance: a review of the literature and a framework for thinking on the notion in information science. Part ii: nature and manifestations of relevance. J. Am. Soc. Inf. Sci. Technol. 58(13), 1915–1933 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institute of Computer ScienceFORTH-ICSHeraklionGreece
  2. 2.Computer Science DepartmentUniversity of CreteHeraklionGreece

Personalised recommendations