Utilizing Entities for an Enhanced Search Experience

This chapter presents a selection of topics, where entities are utilized with the overall aim of improving the users’ search experiences. First, we discuss techniques for assisting users with articulating their information needs, including query assistance services and specialized query building interfaces. Next, we turn to the question of result presentation and introduce entity cards. Finally, we study entity recommendation methods that present users with contextual suggestions, encourage exploration, and allow for serendipitous discoveries.

system assist the user in the process of articulating and expressing her information need? First, we discuss automatic methods that provide users with query suggestions while they are typing their query (Sect. 9.1.1) or after the query has been submitted (Sect. 9.1.2). Then, we present examples of specialized query building interfaces that enable users to formulate semantically enriched (keyword++) queries, by explicitly marking entities, types, or relationships (Sect. 9.1.3).

Query Auto-completion
Query auto-completion (QAC) provides users with query suggestions as they enter terms in the search box. Query auto completion is a common feature in modern search engines; see Fig. 9.1 for an illustration. It helps users to express their search intent as well as to avoid possible spelling mistakes [15]. Most systems rely on the wisdom of the crowds, i.e., suggest completions (matching the entered prefix) that have been most popular among users in the past, based on query logs [5]. Typically, QAC is viewed as a ranking problem, where the aim is "to return the user's intended query at the top position of a list of [candidate] query completions" [15].
Formally, we let q 0 be the incomplete query that the user has typed in so far and q s be a suggested candidate query suffix. Let c(q 0 ⊕ q s ) be the number of times we observe the query q 0 ⊕ q s issued in the query log Q, where ⊕ is the string concatenation operator. The baseline approach to QAC, referred to as most popular completion in [5], is to rank suggestions according to: .
Given that many information needs revolve around entities, entities can be leveraged to improve query auto-completion.

Leveraging Entity Types
Meij et al. [32] focus on a specific subset of queries that can be decomposed into entity and refiner components, such as "ASPIRIN side effects" or "BRITNEY SPEARS video" (refiners typeset in italics). They show that exploiting the type of entities being sought can improve QAC for rare queries. Specifically, we let q 0 = e, where the query entity e is recognized using entity linking techniques (cf. Sect. 7.3). Further, let T e denote the entity types assigned to e in the knowledge base. Their best performing model (called M1 in [32]) looks at the most likely completion for a given entity type y ∈ T e : score(q s ;q 0 ) = P (q s |y) = c(q s ,y) q 0 ⊕q s ∈Q c(q s ,y) , where c(q s ,y) is the number of times we can observe completion q s with an entity of type y in the query log. Note that entities commonly have multiple types assigned to them. Selecting a single "best" type y out of the types of the query entity is an open issue that is not dealt with in [32]. Instead, they evaluate performance for all possible types and then choose the type that led to the best performance on the training set.

Query Recommendations
Unlike "as-you-type" query auto-completion, which assists users during the articulation of their information need, query recommendations (a.k.a. query suggestions) are presented on the SERP once an initial query has been issued. The idea is to help users formulate more effective queries by providing suggestions for the next query. These suggestions are semantically related to the submitted query and either dive deeper into the current search direction or move to a different aspect of the search task [37]. Query suggestions are an integral feature of major web search engines and an active area of research [10,11,18,25,37,43,46]. Figure 9.2 shows query recommendations in action in a major web search engine. Generating query recommendations is commonly viewed as a ranking problem, where given an input query q, the task is to assign a score to each candidate suggestion q , score(q ;q). Like QAC, query recommendation also relies on the wisdom of the crowds by exploiting query co-occurrences and/or click-through information mined from search logs. While log-based methods work well for popular queries, it is a challenge for them to ensure coverage (i.e., provide meaningful suggestions) for rare queries. Most query assistance services perform poorly, or are not even triggered, on long-tail queries, simply because there is little to no historical data available for them. In this section, we will discuss methods that alleviate this problem by utilizing entities in a knowledge base.
We start by presenting the query-flow graph (QFG) method [10] in Sect. 9.1.2.1, which is a seminal approach for generating query recommendations. Then, in Sects. 9.1.2.2-9.1.2.4, we introduce various extensions to the QFG approach that tap into specific characteristics of entities, such as types and relationships. All these methods rely on the ability to recognize entities mentioned in queries. We refer back to Sect. 7.3 for the discussion of techniques for entity linking in queries. The set E q denotes the entities identified in query q.

Query-Flow Graph
The query-flow graph (QFG), proposed by Boldi et al. [10], is a compact representation of information contained in a query log. It is a directed graph G = (V ,E,W ), where the set V of vertices is the distinct set Q of queries contained in a query log, plus two special vertices, v s and v t , representing the start and terminal states of sessions: V = Q ∪ {v s ,v t }. A session is defined as a "sequence of queries of one particular user within a specific time limit" [10]. Commonly, the session time limit is taken to be 30 min. Further, E ⊆ V × V is a set of directed edges and W : E → (0,1] is a weighting function assigning a weight w(q i ,q j ) to each edge (q i ,q j ) ∈ E. Two queries, q i and q j , are connected with a directed edge (q i → q j ), if there is at least one session in the log in which q i and q j are consecutive.
The key aspect of the construction of the query-flow graph is how the weighing function connecting two queries, w(q i ,q j ), is defined. A simple and effective solution is to base it on relative frequencies of the query pair appearing in the query log. Specifically, the weight of the edge connecting two queries is computed as: where c(q i ,q j ) is the number of times query q j follows immediately q i in a session. The normalization coefficient Z is set to such that the sum of outgoing edge weights equals to 1 for each vertex, i.e., j w(q i ,q j ) = 1. Thus, weight w(q i ,q j ) may be seen as the probability that q i is followed by q j in the same search session. This normalization may be viewed as the transition matrix of a Markov chain. Figure 9.3 shows an excerpt from a query-flow graph. Example of a query-flow graph, based on [10]. Note that not all outgoing edges are reported (thus, the sum of outgoing edges from each vertex does not reach 1). The terminal state vertex (v t ) is distinguished using a gray background Based on this graph-based representation, query recommendations can be obtained by performing proximity-based top-k vertex retrieval, either neighborhood-based or path-based. A simple recommendation scheme is to pick, for an input query q, the top-k vertices connected with the largest edge weights. However, as observed by Boldi et al. [10], this method tends to drift off toward popular but unrelated queries. A better recommendation would be to pick the most important query q relative to the initial query q. The recommendation algorithm proposed in [10] is a random walk with restart to a single vertex: a random surfer starts at the initial query q, and at each step either (1) follows one of the outlinks from the current vertex with probability α or (2) jumps back to q with probability 1 − α. This process may also be viewed as applying "a form of personalized PageRank, where the preference vector is concentrated in a single node [vertex]" [10]. More formally, the process is described as computing the transition matrix A of a Markov chain: where P is the row-normalized weight matrix of the query-flow graph, 1 is the identity matrix, and i j is a "one-hot" vector whose entries are all zeroes, except for the j th vector whose value is 1. The parameter α is chosen to be 0.85 in [10]. A has a unique stationary distribution vector v, such that v A = v. This distribution, called the random-walk score relative to q, can be computed using the power iteration method. Then, the highest scoring queries can be returned as the most relevant query suggestions for q. Notably, if the top-scoring query is the termination vertex v t , then it means that the query chain is most likely to end at that point. In that case, it may be wise not to offer any query suggestions to the user.
Instead of using the raw random walk scores, Boldi et al. [10] propose to use a weighting scheme, so as to avoid returning very common queries as suggestions. The variant that yields the best recommendations in their experiments is given by the following formula: where rw q (q ) is the random walk score of the query q with respect to q (personalized PageRank [26]) and rw(q ) is the random walk score of q computed using a uniform preference vector (no personalization, i.e., starting at random at any vertex).
Having introduced the QFG approach, we shall next look at a number of extensions that utilize entities in a knowledge base, in order to improve recommendations for long-tail queries.

Exploiting Entity Aspects
Reinanda et al. [39] define entity aspects as a set of refiners that represent the same search intent in the context of an entity. For example, for the entity BARCELONA F.C., the refiners "live," "live streaming," and "live stream" represent the same search intent, i.e., they amount to one particular aspect of that entity. For each entity e, Reinanda et al. [39] mine a set of aspects A e = {a 1 , . . . ,a m } from a search log. First, the set of queries mentioning that entity, Q e ⊂ Q, is identified using entity linking. Then, refiners r (referred to as query contexts in [39]) are extracted by removing the mention of the entity from the queries. Next, refiners that express the same intent are clustered together. Clustering relies on the pairwise similarity between two refiners, sim(r i ,r j ), which is taken to be the maximum of the following three types of similarity: • Lexical similarity, estimated by the Jaro-Winkler distance between r i and r j . • Semantic similarity, using the cosine similarity of word embeddings of the refiners, cos(r i ,r j ). Specifically, the vector representation of a refiner, r i , is computed as the sum of Word2vec [33] vector of each term within. • Click similarity, obtained using the cosine similarity between the vectors of clicked sites.
For clustering refiners, hierarchical agglomerative clustering with complete linkage is employed. Refiners are placed in the same cluster if their similarity is above a given threshold. By the end of this step, each cluster of refiners corresponds to an entity aspect. The mined aspects are used for query recommendations as follows. During the construction of the query-flow graph, a distinction is made between queries that contain a mention of an entity (i.e., are entity-bearing) and those that are not. For an entity-bearing query, the mentioned entity e and the refiner r are extracted. 1 Then, r is matched against the appropriate entity aspect, by finding the aspect a i ∈ A e that contains r as its cluster member, r ∈ a i . This way, semantically equivalent queries are collapsed into a single entity aspect vertex in the modified query-flow graph. Non-entity-bearing queries are handled as in the regular query-flow graph, i.e., each unique query corresponds to a vertex.
For an incoming new query, the process of generating recommendations works as follows. First, entity linking is performed on the query to decide if it is entitybearing. Then, the query is matched against the appropriate graph vertex (using the same procedure that was used during the construction of the query-flow graph). Finally, recommendations are retrieved from the query-flow graph. 2 Note that entity aspect vertices may contain multiple semantically equivalent queries. In that case, a single one of these is to be selected (for each vertex), e.g., based on query popularity. This approach (referred to as QFG+A) is shown to achieve small but consistent and significant improvements over generic QFG query recommendations.

Entity Types
An obvious limitation of query-flow graphs is that they cannot make recommendations for long-tail or previously unseen queries. Szpektor et al. [46] alleviates this limitation by enhancing query-flow graphs with query templates. 3 Templates are defined by replacing the entity mention in the query by a placeholder, which is an entity type. For example, the queries "New York hotels," "Los Angeles hotels," and "Paris hotels" may be abstracted into a " city hotels" query template. Then, general recommendation rules, like " city hotels" → " city restaurants," may be extracted from query logs. Using such rules, it is possible to generate the recommendation "Yancheng restaurants" for the input query "Yancheng hotels," even if none of those queries have been observed before. Next, we detail the elements of this approach.
Generating Query Templates Each query q in the query log Q is considered for template construction. Given a query, every word n-gram (up to length 3 in [46]) is checked whether it refers to an entity. If yes, then that query segment of the query is replaced with the type(s) of the corresponding entity to produce the corresponding template(s). Here, entity types are not taken to be a flat set but are considered to exist in a hierarchical type taxonomy. We letT e denote the most specific type(s) that entity e is assigned to in the knowledge base. For example, using the DBpedia Ontology as the type taxonomy, the entity ALBERT EINSTEIN has a single most-specific typẽ T e = {Scientist}. By definition, the distance between an entity and (one of) its most specific type(s) is set to 1. Since the type taxonomy is a subsumption hierarchy (cf. Sect. 2.3.1), if entity e is an instance of type y, then it will also be an instance of all supertypes of y. We letT e denote the set of all supertypes of the types inT e . For the example entity ALBERT EINSTEIN,T e = {Person,Agent} (since Scientist is a subtype of Person, which is a subtype of Agent). Thus, the type assignments of an entity are partitioned into most specific types and their supertypes (i.e., T e =T e ∪T e andT e ∩T e = ∅). Further, d(y,y ) is defined as the shortest path distance between types y and y in the type taxonomy. The distance function d(e,y) between entity e and type y is then defined as follows: otherwise . The set of templates constructed from a given query q is denoted as U q . Each template u ∈ U q is associated with a confidence score w(u,q), which expresses how well u generalizes q. Intuitively, the higher a type is located in the type hierarchy, the higher the risk of the corresponding template over-generalizing the query. Thus, the confidence score can be set to be exponentially decaying with the distance between entity e and type y: w(q,u) = α d(e,y) , (9.2) where α is the decay rate (set to 0.9 in [46]). Note that template scoring in Eq. (9.2) does not take into account the uncertainty associated with the type-annotation of the query. The intuition is that by considering the transitions between templates (based on subsequent queries from which they were generated) in a sufficiently large query log, noise will be eliminated and meaningful transition patterns will surface. This will be explained next.

Extending the Query-Flow Graph
In the extended query-flow graph, referred to as query-template flow graph, vertices represent not only queries but templates as well. We let the set U denote all templates that are generated by queries in the search log: U = q∈Q U q . In addition to query-to-query transition edges (E qq ) of the original query-flow graph, we now have two additional types of edges: (1) query-to-template edges (E qu ) and (2) template-to-template edges (E uu ).
• There is a directed edge between query q and template u iff u ∈ U q . The corresponding edge weight w(q,u) is set proportional to the query-template confidence score, which is given in Eq. (9.2). • There is a directed edge between templates u i and u j iff (i) they have the same placeholder type and (ii) there is at least one supporting edge (q i ,q j ) ∈ E qq such that u i ∈ U q i , u j ∈ U q j , and the substituted query segment is the same in q i and q j . The set of all support edges is denoted as E s (u i ,u j ). Then, the edge weight between u i and u j is set proportional to the sum of edge weights of all supporting query pairs: For example, for the template-to-template edge " city hotels" → " city restaurants," the set of support edges includes {"Paris hotels" → "Paris restaurants," New York hotels" → "New York restaurants," . . . }.
Normalization is performed in both cases to ensure that the outgoing edge weights of graph vertices sum up to 1.

Generating Query Recommendations
Using the regular query-flow graph, candidate query recommendations q for an input query q would be those for which there exists a directed edge (q,q ) ∈ E qq . With the extended query-template flow graph, candidate recommendations also include queries that have not been observed in the logs before but can be instantiated via a template. Specifically, there needs to be a mapping edge (q,u) ∈ E qu and a template-to-template transition edge (u,u ) ∈ E uu , such that by instantiating u with the entity extracted from q, it yields q as the result. Then, candidate recommendations are scored according to the following formula: where ins(u ,q,u) denotes the query that is the result of instantiating template u based on query q and template u. Given that edge weights are normalized, the resulting score will be in the range [0,1] and "can be interpreted as the probability of going from q to q by one of the feasible paths in the query-template flow graph" [46].

Entity Relationships
Bordino et al. [12] extend the query-flow graph with entity relationship information, and Huang et al. [25] capitalize on this idea for generating query recommendations. Suppose that two queries q i and q j appear in the same session and they mention entities e i and e j , respectively. Then, in addition to the flow from q i to q j in the query-flow graph, we can also utilize the relationships between e i and e j in the knowledge base. More formally, the enhanced graph, referred to as EQGraph in [12], has query vertices V Q and entity vertices V E , with query-to-query edges E qq , entity-to-query edges E eq , and entity-to-entity edges E ee . See Fig. 9.4 for an illustration.

Entity-to-Query Edges
Each entity is connected to all the queries that mention it. The edge between entity e and query q is given a weight proportional to the relative frequency of that query in the log: where c(q) is the number of times q appears in the query log, and E q denotes the set of entities that are extracted from q via entity linking. Note that the outgoing edge weights sum up to one for each entity vertex.

Entity-to-Entity Edges
Edge weights w(e,e ) represent the transition probability between a pair of entities e and e . Bordino et al. [12] derive these weights based on the query log, by considering all query-to-query transitions q → q , where q mentions e and q mentions e : This formulation distributes the probability mass uniformly among the possible (|E q | × |E q |) entity-to-entity transitions derived from q → q .

Generating Query Recommendations
Huang et al. [25] generate query recommendations by computing personalized PageRank [26] on the EQGraph, starting from entities. The core of their approach lies in the idea that instead of considering the entities that are mentioned in the query (E q ), they consider related entities from the knowledge graph. This set of related entities, denoted as E R , is derived based on the notion of meta-paths [45]. A meta-path M in the knowledge graph is a sequence of entity types y 1 , . . . ,y n connected by predicates (i.e., relationship types) p 1 , . . . ,p n−1 , such that M = y 1 Each of these meta-paths represents a specific direct or composite ("multi-hop") relationship between two entities. Two entities may be connected by multiple meta-paths; a natural approach, followed in [25], is to select the shortest meta-path between them to represent their relationship. Let M be the set of meta-paths over the entity types in the KG, and M y ⊂ M be the set of outgoing meta-paths for type y. Related entities are collected by performing a path-constrained random walk [29] on knowledge graph predicates, with each meta-path M ∈ M y , for each of the types associated with the linked entities in the query (y ∈ T e ,e ∈ E q ). Each of these related entities e ∈ E R accumulates weight, w(e), based on the various meta-paths it can be reached on. See Algorithm 9.1 for the detailed procedure.
The most relevant queries, with respect to the related entities E R , are returned as recommendations. Specifically, for each of the related entities, e ∈ E R , personalized PageRank is performed on the EQGraph, starting from e with initial probability Algorithm 9.1: Related entity finding for query recommendation [25] Input: query-flow graph, knowledge graph, query q Output: related entities E R with weights w Note that this recommendation method considers only the entities mentioned in the query but not the other contextual terms in the query. It means that if two queries q and q mention the same entities (E q = E q ) then generated recommendations will be exactly the same for the two.

Query Building Interfaces
In Chap. 4, we have seen that leveraging semantically enriched queries, referred to as keyword++ queries, yields improved retrieval performance. Such keyword++ queries may contain annotations of specific entities, target types, or relationships. One way to obtain those annotations is via automated techniques aimed at query understanding-which we have discussed in Chap. 7. Alternatively, it may be delegated to the user to provide semantic annotations for queries, and thereby more explicitly express the underlying information need. This, however, can be challenging for ordinary users, due to their unfamiliarity with the underlying knowledge base. Furthermore, even those that are acquainted with the knowledge base will find it problematic to navigate the large space of entities, types, and relationships without some support. In order to aid users in the process of formulating complex queries, specialized query building interfaces have been proposed [6,24,42]. A common feature of these systems is that they provide context-sensitive suggestions. The STICS system offers suggestions for entities and categories, as users type query terms; a screenshot is shown in Fig. 9.5. Schmidt et al. [42] present a corpus- adaptive extension, where the ranking of candidate suggestions also takes into account the underlying document collection. That is, they only suggest entities and categories "that lead to non-empty results for the document collection being searched" [42]. They further introduce a data structure for storing pre-computed relatedness scores for all co-occurring entities, in order to keep response times below 100 ms. Another example is the Broccoli system, which targets expert users and allows them to incrementally construct tree-like SPARQL queries, using similar techniques (i.e., corpus-based statistics) for generating suggestions [6]. Figure 9.6 presents a screenshot of the system.

Entity Cards
In recent years, there has been an increasing trend of surfacing structured results in web search in the form of various knowledge panels. Being served with rich and focused responses, users no longer need to engage with the organic (documentoriented) search results to find what they were looking for. This marks a paradigm shift in search engines evolving into answer engines. One group of knowledge panels, often referred to as direct displays, provide instant answers to a range of popular information needs, e.g., weather, flight information, definitions, or how-to questions. Some direct displays invite users to engage and interact with them (like currency conversion or finance answers), while others yield a clear inline answer (such as dictionary or reference answers) with no further interaction expected. Our focus in this section will be on another type of knowledge panel, called entity card, which summarizes information about a given entity of interest. Unlike direct displays, whose mere goal is to provide a succinct answer, entity cards intend to serve an additional purpose-to present the user with an overview of a particular entity for contextualization and further exploration.
An entity card portrays a summary of a selected entity, commonly including the entity's name, type, short description, a selection of key attributes and relationships, and links to other types of relevant content.
Entity cards are an integral component of modern search engine result pages, on both desktop and mobile devices [14,28]. Triggered by an entity-bearing query, a rich informational panel is displayed (typically on the top-right of the SERP on a desktop device), offering a summary of the query entity, as shown in Fig. 9.7a, b. It has been long known that providing query-biased snippets for documents in the result list positively influences the user experience [49]. Entity cards may be viewed as the counterpart of document snippets for entities, and, as we shall show in this section, may be generated in a query-dependent fashion. It has been shown that entity cards attract users' attention, enhance their search experience, and increase their engagement with organic search results [14,36]. Furthermore, when cards are relevant to their search task, users issue significantly fewer queries [14] and can accomplish their tasks faster [36].
We shall begin with an overview of what is contained in an entity card. Then, we will focus on the problem of selecting a few properties from an underlying knowledge base that best describe the selected entity, with respect to a given query, which will serve as the factual summary of that entity.

The Anatomy of an Entity Card
Entity cards are complex information objects that are dynamically generated in response to an entity-oriented query, after determining the intended entity for that query (cf. Sect. 7.3). Figure 9.8 shows a card layout that is commonly used in contemporary web search engines, comprising (1) images, (2) the name and type of the entity, (3) a short textual description, (4) entity facts, and (5) related entities. Additionally, depending on the type of the entity and the intent of the search query, other components may also be included, such as maps, quotes, tables, or forms.
Most elements of entity cards have their own set of associated research challenges. An entity may be associated with multiple types in the KB. For example, the types of ARNOLD SCHWARZENEGGER in Freebase include, among others, tv.tv actor, sports.pro athlete, and government.politician. The problem of selecting a single "main" type to be displayed on the card has been addressed using both context-independent [7] and context-dependent [50] methods. For emerging entities, that already have some facts stored about them in the KB, but lack a Wikipedia-style summary, natural language descriptions may be produced automatically [17,41]. The factual summary, a truncated view of facts about the entity, is a central element of entity cards. We devote the remainder of this section to this very problem. Finally, related entity suggestions typically utilize search logs and entity co-occurrence information; we shall discuss specific methods in Sect. 9.3. Clicking on one of the related entities typically launches a new query with the related entity.

Factual Entity Summaries
The problem of generating informative entity summaries from RDF data has generated considerable interest over the recent years [16,22,23,47,48]. Since descriptions of entities in a knowledge base typically include hundreds of factual statements, "for human users it becomes difficult to comprehend the data unless a selection of the most relevant facts is provided" [47]. Below, we present the approach by Hasibi et al. [23] that is shown to be more effective than other relevance-oriented fact ranking methods, which employ variations of the PageRank algorithm [16,47,48]. Notably, it considers facts with both entity and literal object values.
Let F e denote all the facts stored about a given entity e in the knowledge base K. That is, F e consists of all predicate-object pairs out of those SPO triples, where e appears as subject: For notational convenience, we shall use the shorthand f to denote a single fact, which is essentially a property-value pair; further, we shall write p f and o f to denote the predicate and object elements of the fact, respectively. Since F e is typically large (on the order of hundreds), the challenge is to select a small subset of them, to be displayed in the summary section of the entity card, that are deemed to have the highest utility for the user. Hasibi et al. [23] argue that factual entity summaries serve a dual purpose: "they offer a synopsis of the entity and, at the same time, can directly address users' information needs." Therefore, when selecting which facts to include in the summary, one should consider both their general importance, irrespective of any particular information need, as well as their relevance to the given user query. These two quantities are combined under the notion of utility. The utility of fact f is defined as a linear combination of its general importance and its relevance to the user query q: For simplicity, importance and relevance are given equal weights in [23]. The generation of factual summaries is addressed in two stages. First, facts are ranked according to their utility. Then, the top-ranked facts are visually arranged in order to be displayed on the entity card.

Fact Ranking
The ranking of facts is approached as a learning-to-rank problem, using two main groups of features, aimed at capturing either the importance or the relevance of a fact. To learn the ranking function, target labels are collected via crowdsourcing for each dimension separately on a 3-point scale (0..2). Then, the two are combined with equal weights (cf. Eq. (9.3)), resulting in a 5-point scale (0..4). Below, we shall introduce some of the most effective features developed for relevance and importance, respectively. Table 9.2 provides an overview. For a complete list of features, we refer to [23].
Importance features are mostly based on statistics derived from the knowledge base. We introduce the concepts of fact frequency and entity frequency, which are loosely analogous to collection frequency and document frequency in document retrieval. Specifically, the fact frequency of object is the number of SPO triples in the KB with a given object value:  Entity frequency of predicate is the number of entities in the KB that have at least one fact associated with them with a given predicate: Another type-aware variant of the above statistic considers only those entities that, in addition to having the given predicate, also are of a given type y: Furthermore, we define the type frequency of a predicate to be: With the help of these statistics, we define the following features for measuring the importance of fact f for entity e: • Normalized entity frequency of predicate is the relative frequency of the fact's predicate across all entities: where |E| is the total number of entities in the KB. • Type-based importance also considers the frequency of the fact's predicate, but with respect to the types of the entity. Following [52], it is estimated using: where |T | is the total number of types in the KB.
• Predicate specificity aims to identify facts with a common object value but a rare predicate: For example, the fact (capital, OTTAWA) would have relatively high predicate specificity, given that the object is frequent, while the predicate is relatively rare. • IsEntity is a binary indicator that is true if the fact's object is an entity.
Relevance features capture the relevance of a fact with respect to the search query.
• Lexical similarity is measured by taking the maximum similarity between the terms of the fact's object and of the query using: where dist() is a string distance function, taken to be the Jaro edit distance in [23]. • Semantic similarity aims to address the vocabulary mismatch problem, by computing similarity in a semantic embedding space. Specifically, we compute the average cosine similarity between terms of the fact's object and of the query: where t denotes the embedding vector corresponding to term t. Hasibi et al. [23] use pre-trained Word2vec [33] vectors with 300 dimensions. The denominator is the multiplication of the number of unique terms in the fact's object and in the query, respectively. • Inverse rank promotes facts with an entity that is highly relevant to the query as the object value. Entities in the KB are ranked with respect to the query. Then, where E k (q) is the set of top-k ranked entities returned in response to q, and rank() returns the position of an entity in the ranking (or ∞ if the entity cannot be found among the top-k results). • Context length is the number of query terms that are not linked to any entity: where linked(q) denotes the set of query terms that are linked to an entity.

Summary Generation
The ranked list of facts we just obtained is yet to be arranged into a summary that can be presented to the user. Visually, a summary consists of a number of lines, each subdivided into heading and value parts. Additionally, it has a maximum display size, defined in terms of the maximum number of rows (τ h ) and the maximum number of characters within each row (τ w ); see Fig. 9.9. A straightforward approach is just to fill this structure with the top-ranked facts, by using the predicate label from the KB as the heading and the subject as the value part in each line. There are, however, some additional considerations that, if addressed, can yield higher quality summaries.
• There might be semantically identical predicates, even within a single KB, e.g., <foaf:homepage> and <dbp:website> in DBpedia. Such duplicates need to be identified and resolved, such that only one of them is included in the summary. • There may be multiple facts with the same predicate, e.g., parents of a person or founders of a company. While these constitute separate facts, the object values of these so-called multi-valued predicates can be concatenated together into a single list for display purposes.
Hasibi et al. [23] address these issues with a summary generation algorithm, shown in Algorithm 9.2. Input to this algorithm is the list of top-k facts, generated by the fact ranking step, denoted asF e . The first line of the summary generation algorithm creates a mapping from predicates inF e to their human-readable labels; these are commonly provided as part of the KB schema. Predicates that are mapped to the same label are then recognized as semantically identical. The mapping function may implement additional heuristics, specific to the underlying knowledge base.

Entity Recommendations
Earlier in this chapter, we have discussed tools that help users with expressing their information needs and with getting direct answers to those needs. There are also situations where users' information goals are less clearly defined, and they would just like to browse and explore, without looking for anything specific. Examples include learning about people in the news or exploring future travel destinations. Therefore, in addition to traditional search assistance tools, such as query suggestions and direct answers, exploration and discovery should also be regarded as central elements of next-generation search systems [55]. This section presents recommendation techniques that enable exploration, with the goal of increasing user engagement.
Specifically, our objective is to provide related entity recommendations (a.k.a. related entity suggestions) to users, based on their context. We shall consider multiple contexts that may serve as input data: (1) a particular entity (Sect. 9.3.1), (2) the user's current search session (Sect. 9.3.2), and (3) a given entity in a particular text context (Sect. 9.3.3).
The entity recommendation task is approached as a ranking problem: given some context (e.g., a particular entity or a search session) as input, return a ranked list of entities e 1 , . . . ,e k from an entity catalog E that are related to the user's context.
How does this problem relate to other tasks that have been discussed earlier in this book? A core component underlying all recommendation methods is a measure of relatedness between an input entity and a candidate entity. Pairwise entity relatedness has already been used in entity linking, for entity disambiguation (cf. Sect. 5.6.1.3), and those methods are applicable here as well. Another related task is that of finding similar entities (cf. Sect. 4.5), which also boils down to a pairwise entity measure. The similar entity finding task, however, has a different objectiveit aims to complete an input set of entities, with similar entities. Consequently, it measures pairwise entity similarity as opposed to entity relatedness.
The degree of entity relatedness may be estimated using simple measures of statistical association, based on entity co-occurrence information (e.g., in search logs or in Wikipedia articles). Another family of methods makes use of entity relationship information stored in knowledge graphs and employs graph-theoretic measures (e.g., PageRank). Yet another group of approaches infers relatedness based on the content (i.e., attributes or descriptions) of entities.
In addition to receiving entity recommendations, users may also be interested in finding out why certain suggestions were shown to them. The problem of explaining recommendations boils down to the task of generating a human-readable description of the relationship between a pair of entities. This is discussed in Sect. 9.3.4.

Recommendations Given an Entity
We start by discussing the case of recommending entities related to a given input entity. A common application scenario is that of entity cards in web search, which are triggered by an entity-bearing query. These cards often include a "People also search for" section, displaying entities that are related to the query entity; see Fig. 9.8. This task may be formalized as the problem of estimating the probability of a candidate entity e , given an input entity e, P (e |e).
Blanco et al. [9] present the Spark system (with previous versions of the system described in [27,58]), which had been powering related entity suggestions in Yahoo! Web Search. Spark extracts several signals (over 100 features) from a variety of proprietary and public data sources (including Yahoo!'s knowledge graph and web search logs, and social media platforms Flickr and Twitter) and combines them in a learning-to-rank framework. The training data consists of 47K entity pairs, labeled by professional editors on a five-point scale. Aggarwal et al. [2] show that comparable accuracy may be achieved by utilizing only publicly available data, in particular, Wikipedia, and using only 16 features. We distinguish between four main groups of features: co-occurrence features, graph-theoretic features, content-based features, and popularity features. Popularity features and graph-theoretic features are unary, expressing the importance of an entity on its own. The remaining features are binary, capturing the strength of association between two entities. Table 9.3 presents a non-exhaustive selection of features.
Co-occurrence features Intuitively, entities that are observed to occur frequently together are likely to be related to each other. One question here is what data collection C to use for extracting co-occurrence information. Another question is what co-occurrence statistic to compute based on those observations. Prior work has considered a wide variety of data sources, including search logs [9], web pages [3], Wikipedia [2,35,44], Twitter [9], and Flickr [9]. Co-occurrence measures include joint, conditional, and reverse conditional probabilities, pointwise mutual information, KL divergence, entropy, and the Wikipedia link-based measure (WLM) [35]. Graph-theoretic features The most commonly used feature in this group is the PageRank score of an entity in the knowledge graph. PageRank may also be computed on a hyperlink graph obtained from the Web [9]. For details on PageRank and for additional centrality measures, we refer back to Sect. 4.6. Content-based features This set of features aims to capture the similarity between a pair of entities based on their content. A standard approach is to represent entities either as term vectors or embedding vectors, and then compute the cosine similarity of those vectors. See Sect. 4.5.1 for alternative ways of measuring pairwise entity similarity.

Popularity features
Popularity is based on the frequency of an entity in a given data source, e.g., search queries and sessions, or number of views or clicks in web search. Additional popularity features were discussed in Sects. 4.6.1 and 5.6.1.1.

Personalized Recommendations
Rather than suggesting entities related to a given input entity, in this section we discuss methods that provide personalized entity recommendations based on the user's current search session. A number of approaches have been proposed for learning models for specific domains, such as movies or celebrities [8,57]. Such model-based methods rely on manually designed domain-dependent features, related to specific properties of entities (e.g., the genre of a movie or how many pop singers were viewed by a specific user). There is an obvious connection to make here to traditional item-based recommender systems (e.g., the ones used in e-commerce systems), such as collaborative filtering [19]. One main difference is that in collaborative filtering approaches the user-item matrix is given. For entity recommender systems, user preferences of entities are more difficult to observe. Another difference is the sheer scale of data (i.e., millions of entities vs. thousands of products in an e-commerce scenario) coupled with extreme data sparsity. Fernández-Tobías and Blanco [20] perform personalized entity recommendations using a purely collaborative filtering approach. Inspired by nearest neighbor techniques, these memory-based methods exploit user interactions that are available in search logs. Since they do not depend on descriptions or properties of entities, memorybased methods generalize to arbitrary types of entities and scale better to large amounts of training data than model-based methods. Next, we present three probabilistic methods for estimating P (e |s), the probability of recommending entity e to a user based on her current search session s. 4 These methods are named after how item-to-item similarity aggregation is performed: entity-based, query-based, or session-based. In their paper, Fernández-Tobías and Blanco [20] define multiple alternatives for each component of these models. Here, we will discuss a single option for each, the one that performed best experimentally. According to the results reported in [20], the entity-based method performs best, followed by the query-based and then the session-based approaches.
We shall use the following notation below. Let Q be the set of unique queries in the search log and S be the set of all user sessions. For a given session s ∈ S, Q s ⊂ Q denotes the set of queries issued and E s ⊂ E denotes the set of entities clicked by the user within that search session.

Entity-Based Method
The intuition behind the first method is that an entity e is more likely to be relevant to the user's current session s if it is similar to other entities that have previously been clicked by the user in the same session. Formally, this is expressed as: where P (e |e) captures the similarity between a pair of entities and P (e|s) expresses the relevance of e given the search session s. Pairwise entity similarities are estimated in a collaborative fashion, by measuring the co-occurrence of entities within all user sessions using the Jaccard coefficient. To estimate the relevance of a clicked entity e within a session s, we aggregate the importance of e for each query q in that session, weighted by the query likelihood in that session: where dwell(e,q,s) is the time spent on examining entity e for query q in session s. In case the user did not click on e as a result to q, P (e|q,s) is taken to be 0. The probability P (q|s) in Eq. (9.4) captures how important that query is within its session. It tends to reason that more recent queries should be considered more important, as they represent more accurately the user's current interests (which may have shifted over time). This notion of temporal decay is formally expressed as: (9.6) where e is the mathematical constant (the base of the natural logarithm), t q is the timestamp of query q, and t s is the timestamp of the last query in the session.

Query-Based Method
The second approach works by first identifying queries from other sessions in the search log that are potentially relevant to the current session. Then, it retrieves entities from those sessions. Formally: where P (q|s) is the probability that query q is relevant to the current session s and P (e |q) measures how relevant e is for query q (across all sessions). Note that, in contrast to the entity-based method, the queries q we aggregate over are not present in the current session. Rather, these are chosen from the queries submitted by other users, performing similar tasks. The relevance of an entity given a query is estimated by considering all sessions in the search log that contain that query: P (e |q) ∝ s ∈S P (e |q,s)P (q|s)P (s) , (9.8) where as before, P (e |q,s) is measured using dwell time (cf. Eq. (9.5)) and P (q|s) is estimated based on temporal decay (cf. Eq. (9.6)). Note that P (q|s) here expresses the probability of choosing the query q from session s containing that query (q ∈ Q s ) and is not to be confused with P (q|s) in Eq. (9.7), where it is used to capture the relevance of a query that is not observed in the given session (q / ∈ Q s ). Finally, P (s) is assumed to be uniform for simplicity.
The query relevance probability, P (q|s) in Eq. (9.7), is defined to select queries from the search log (q ∈ Q \ Q s ) that are similar to the ones in the current session: where P (q|q ) expresses the similarity between a pair of queries and is computed based on the co-occurrence of q and q within all sessions in the search log using the Jaccard coefficient. As before, P (q |s) uses the temporal decay estimator (cf. Eq. (9.6)).

Session-Based Method
The last approach works by finding sessions similar to the current session, then recommending entities from those sessions: where P (e |s ) is the importance of an entity given a session, computed as given by Eq. (9.4). The pairwise session similarity, P (s |s), is estimated based on entity embeddings. Specifically, Fernández-Tobías and Blanco [20] use Word2vec [33] (where sessions correspond to documents and entities within sessions correspond to words within documents) and extract embedding vectors of dimension 100. The Fig. 9.10 An example of contextual entity recommendations. Image is based on [31] similarity between two sessions is then computed based on the distance between the centroids of entities within them: where e is the embedding vector corresponding to entity e.

Contextual Recommendations
Web search is a prominent application area for entity recommendations but is certainly not the only one. Entity recommendation may also be offered directly within the application where content is consumed. As one such example, Lee et al. [31] present the scenario of a user reading a document on a tablet or e-reader device. At some point, the user might stumble upon an entity that she wishes to learn more about. Instead of leaving the application and switching to a web search engine to query for that entity, the user might just highlight and tap on an entity of interest. She will then be presented with a list of contextually relevant entities, as it is shown in Fig. 9.10.
According to the outlined scenario, the input to the contextual entity recommendation problem consists of an input entity e and some context c. Specifically, the  [31]. The input entity (e) vertex is marked black, contextual entity vertices (E c ) are marked gray context is a window of text around the selected entity mention (100 terms before and after in [31]). The approach proposed by Lee et al. [31] consists of three main steps, which are illustrated in Fig. 9.11. 1. A focused subgraph is extracted from the underlying knowledge graph G. The vertices of this focused subgraph are V = {e}∪E c ∪E c , where e is the input entity, E c is the set of context entities, recognized in c by performing entity linking, and E c is the set of entities reachable from E c via paths of length one in the knowledge graph. The edges between these vertices are induced from G. 2. Each candidate entity e in the focused subgraph is scored using two different methods: context-selection betweenness and personalized random walk. Contextselection betweenness (CSB) captures the extent to which the candidate entity e serves as a bridge between the input entity e and the context entities E c . Intuitively, a higher CSB score means that the candidate entity plays a more important role in connecting the input and context entities. Formally, CSB considers all shortest paths between the input and context entities, which go through the candidate entity: The threshold γ is used for emphasizing context entities that are semantically related to the input entity (γ = 0.5 in [31]). The normalization factor is set to: The other scoring method is personalized random walk (PRW, a.k.a. personalized PageRank [26]), which aims to measure the relevance of entities given the user's selection. To compute these scores, the random jump probabilities are initialized as follows. The input entity vertex is assigned probability 0 < x e < 1, the context entity vertices are assigned probability x c /|E c |, where 0 ≤ x c ≤ x e , and all other entity vertices are assigned probability 0. Lee et al. [31] use x e = 0.05 and x c = 0 in their experiments, and report that x c > 0 does not lead to significant improvements. 3. The final score for each entity vertex is computed by taking a weighted combination of the context-selection betweenness and personalized random walk scores: where |V | is the number of vertices in the focused subgraph and α is a scaling factor. The multipliers serve normalization purposes, making the two scores compatible.

Explaining Recommendations
Thus far in this section, we have presented both non-personalized and personalized methods for recommending related entities (e ) given an input entity (e). In addition to the recommendations themselves, users might also be interested in finding out why certain entities were shown to them. This brings us to answering the question: How are the input entity and the recommended entity related? Such explanations are offered, e.g., on entity cards in modern web search engines by hovering the mouse over a recommended entity; see Fig. 9.12. Another typical application scenario for explaining entity relationships is timeline generation [3]. One of the earliest works addressing the problem of explaining the relationship between two entities is the dbrec music recommendation system [38]. It offers explanations in the form of a list of shared property-value pairs between the input and recommended entities, as shown in Fig. 9.13. This form of presentation, however, was considered as "too geeky" by 6 out of the 10 test subjects participating in the user evaluation [38]. Instead, human-readable descriptions that verbalize the relationship are more natural to use. The use of natural language has also shown to improve confidence in decision-making under uncertainty [21]. Three main lines of approaches have been proposed in prior work for generating natural language descriptions of entity relationships: (1) by manually defining templates [3], (2) by retrieving sentences from a text corpus [54], and (3) by automatically creating templates for a specific relationship and then filling the template slots for a new relationship instance [53]. Below, we briefly elaborate on the latter two.
All existing approaches solve a simplified version of the task of explaining entity relationships, by focusing on a specific relationship between a pair of entities. This corresponds to generating a textual description for an SPO triple, where the subject is e, the predicate is p, and the object is e . We shall refer to the triple (e,p,e ) as relationship instance. When referring to predicate p, we shall use the terms predicate and relationship interchangeably. As it is illustrated in Fig. 9.13, entities may be connected via multiple relationships. Selecting p from the set of possible predicates that connect two entities remains an open research challenge to date.

Explaining Relationships via Sentence Ranking
Voskarides et al.
[54] approach the task as a sentence ranking problem: automatically extracting sentences from a text corpus and ranking them according to how well they describe a given relationship instance (e,p,e ). First, a set X of candidate sentences is extracted from a corpus of documents. In [54], this corpus is Wikipedia. Other document collections may also be used, as long as documents are pertinent to the entities of interest. A sentence is considered as a candidate if (1) it originates from the Wikipedia page of e or e and contains a mention to the other entity or (2) it mentions both e and e . In order to make sentences self-contained outside the context of the source document, pronouns "she" and "he" are replaced with the name of the respective entity. Further, sentences are annotated with entities by performing entity linking. As an illustration, consider the sentence "He gave critically acclaimed performances in the crime thriller Seven. . . ," which, after these enrichment steps, becomes "BRAD PITT gave critically acclaimed performances in the crime thriller SEVEN. . . " Next, candidate sentences x ∈ X are ranked using supervised learning. Four groups of features are employed: • Textual features consider the importance of the sentence on the term level.
These include sentence length, aggregated IDF scores, sentence density [30], and fractions of verbs, nouns, and adjectives. • Entity features characterize the sentence based on the mentioned entities. These include, among others, whether e and e are linked in x, and the distance between the positions of their mentions. Another group of features focuses on other entities mentioned in the sentence and whether those are related to e and e . Fig. 9.14 Dependency graph for the sentence "Brad Pitt appeared in the American epic adventure film Troy," using entities and predicates from DBpedia • Relationship features indicate whether the relationship p occurs in x. Exact termbased matching has low coverage (e.g., "spouse" vs. "husband" or "married"), therefore both synonym-based matches (using Wordnet) and word embeddings (using Word2vec [33]) are considered. • Source features describe the position of x and the number of occurrences of e and e in the document from which x originates.
[54] train their models on a set of manually annotated sentences, using a five-level graded relevance scale. They show that learning relationshipspecific models, as opposed to a single global model, can yield additional improvements.

Generating Descriptions of Relationships
The previous approach is limited by the underlying text corpus, which may not contain descriptions for certain relationship instances. Voskarides et al. [53] propose to overcome this by automatically generating descriptions. The idea is to learn how a given relationship p is typically expressed (in the document corpus), and create sentence templates for that relationship. Then, for a new relationship instance, the appropriate template can be instantiated. As before, it is assumed that a given relationship between two entities can be expressed as a single sentence. This sentence should mention both e and e , and possibly other entities that may provide additional contextual information. The following example sentence is given as an illustration in [53] for the (BRAD PITT, stars in, TROY) relationship instance: "Brad Pitt appeared in the American epic adventure film Troy." It not only verbalizes the "stars in" predicate but also mentions other entities and attributes (the film's genre and origin) to offer additional context. To be able to provide such contextual information, each sentence is augmented with an entity dependency graph. In this graph, vertices represent entities and edges represent relationships (predicates). The graph is created by traversing paths in the knowledge base between each pair of entities that are mentioned in the sentence. See Fig. 9.14 for an illustration. 6 6 In their paper, Voskarides et al. [53] use Freebase as the underlying knowledge base and pay special attention to compound value type (CVT) entities. CVT entities are specific to Freebase, and are used for modeling attributes of relationships (e.g., date of a marriage). For ease of presentation, we will not deal with CVT entities in our discussion.
The template creation process then takes as input, for each predicate, a set of relationship instances, sentences describing those relationship instances, and entity dependency graphs corresponding to those sentences. The following sequence of steps are performed: 1. Entities sharing the same predicates are clustered together across the dependency graphs. This will group entities of the same type, such as persons and films. 2. A compression graph G C is created from sentences where vertices are either words or entity clusters. 3. G C is traversed for finding valid paths between all pairs of entity cluster vertices.
A path is considered valid if (i) it contains a verb and (ii) it can be observed as a complete sentence at least once in the corpus. 4. A template is constructed from each path, which is supported by a minimum number of sentences in the corpus.
With a set of templates at hand, generating a description for a new relationship instance (e,p,e ) goes as follows. First, the available templates for predicate p are ranked. Two template scoring functions are presented in [53], one based on cosine similarity of TF-IDF term vectors and another using feature-based supervised learning. The top-ranked template is then instantiated by filling its slots with entities from the knowledge base. If multiple instantiations exist, then one of those is chosen randomly. If the template cannot be instantiated, then we proceed to the next template in the ranking.

Summary
This chapter has introduced search assistance tools that help users (1) express their information needs, (2) get direct answers, and (3) explore related content with the help of entities. We have started with query assistance features, such as query autocompletion and query recommendation, which users would expect today as standard functionality from a modern search engine. An issue with traditional methods, which rely solely on query logs, is that of coverage. That is, they fail to provide meaningful suggestions for long-tail queries. We have discussed how knowledge bases may be utilized to alleviate this problem, yielding small but significant improvements over traditional methods. Next, we have looked at entity cards, a new type of search result presentation that has been adopted by major web search engines and intelligent personal assistants. Each card presents a concise summary of a specific entity, and can satisfy the user's information need directly, while also encouraging further engagement with search results and exploration of related content. We have addressed, in detail, the question of which facts to highlight about an entity in the limited screen space that is available on the card. Finally, we have presented techniques for promoting exploratory search by recommending related entities to users. We have further discussed how to generate a natural language explanation for the relationship between an input entity and a recommended related entity.

Further Reading
Web search result pages are becoming increasingly complex, featuring direct displays, entity cards, query suggestions, advertisements, etc., arranged in a nonlinear page layout. With these novel interfaces, determining the user's satisfaction with the search results is becoming more difficult. Research in this area includes the topics of evaluating whole page relevance [4], understanding how users examine and interact with nonlinear page layouts [13,36], and detecting search satisfaction without clicks [28,56]. Entity cards are the most widely used and universally applicable tools for summarizing entity information, but there are other possibilities that could serve users better in certain application scenarios. One such alternative that has garnered research interest is entity timelines, which organize information associated with an entity, arranged along a horizontal time axis. Timeline visualizations are often coupled with interactive features to enable further exploration. For example, Rybak et al. [40] visualize how a person's expertise changes over the course of time. Tuan et al. [51] and Althoff et al. [3] generate a timeline of events and relations for entities in a knowledge base.
In this chapter, we have focused on the algorithmic aspects of generating entity recommendations. For a study on how people interact with such recommendations, see, e.g., [34]. The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.