U-Measure

Definition

U-measure is a versatile evaluation measure. For a given textual information access task, U-measure first builds a trailtext, which is a concatenation of all the texts that the user has read (which may be constructed under some user behaviour model or based on some real evidence such as clicks and eyetracking or mousetracking data). Then, U-measure computes a score using a decay function based on the positions of relevant items within the trailtext. Formally, a trailtext tt is a concatenation of n strings: tt = s₁s₂…s_n. The strings may be documents, parts of documents, snippets, sentences, or any other fragments of text that have been read. The offset position of s_k(1 ≤ k ≤ n) is defined as $\textit {pos}(s_{k})=\sum _{j=1}^{k}|s_{j}|$, where the length of each string is measured in terms of the number of characters. Furthermore, the position-based gain is defined as g(pos(s_k)) = 0 if s_k is considered nonrelevant and g(pos(s_k)) = gain_l if its relevance level is l. For...