# Analyzing spatial data from mouse tracker methodology: An entropic approach

- 1k Downloads
- 3 Citations

## Abstract

Mouse tracker methodology has recently been advocated to explore the motor components of the cognitive dynamics involved in experimental tasks like categorization, decision-making, and language comprehension. This methodology relies on the analysis of computer-mouse trajectories, by evaluating whether they significantly differ in terms of direction, amplitude, and location when a given experimental factor is manipulated. In this kind of study, a descriptive geometric approach is usually adopted in the analysis of raw trajectories, where they are summarized with several measures, such as maximum-deviation and area under the curve. However, using raw trajectories to extract spatial descriptors of the movements is problematic due to the noisy and irregular nature of empirical movement paths. Moreover, other significant components of the movement, such as motor pauses, are disregarded. To overcome these drawbacks, we present a novel approach (EMOT) to analyze computer-mouse trajectories that quantifies movement features in terms of entropy while modeling trajectories as composed by fast movements and motor pauses. A dedicated entropy decomposition analysis is additionally developed for the model parameters estimation. Two real case studies from categorization tasks are finally used to test and evaluate the characteristics of the new approach.

## Keywords

Mouse tracking Spatial data Entropy analysis Movement trajectories Aimed movements## Introduction

Human movement is an integral part of the perceptual-motor system. People constantly use their arms, hands, and fingers in simple and complex activities, as for example, reaching objects, pointing at visual targets, typing on a keyboard, or moving a computer mouse. In all these activities, movements can be very complex and their study, in terms of kinematics and/or simple trajectories, can provide a simple and valid basis to uncover cognitive and behavioral processes underlying the observable structure of the data. Recently, many authors have focused on the analysis of human movements measured by means of a computer mouse, an approach called mouse tracking methodology (e.g., see: Freeman & Ambady, 2010; Hehman, Stolier, & Freeman, 2015). The basic idea is to consider the collected mouse movement trajectories as a continuous source of real-time information on the internal cognitive processes activated during a particular experimental task. For instance, in a dichotomous categorization task where two alternative choices are presented on a computer display (e.g., target vs. distractor), mouse trajectories can reveal, according to the experimental manipulation, the eventual competitive attraction that one of the two choices (e.g., distractor) acts on the correct target category (Spivey and Dale 2006). This simple idea has recently been applied in several research fields, including social categorization (Dale et al. 2007), moral decision-making (Koop 2013), language comprehension (Morett and Macwhinney 2013), numbers representation (Faulkenberry 2014), and cross-modal processing (Coco and Duran 2016).

Several strategies are available to analyze mouse tracking data both from a *model-dependent* framework (e.g., Friedman, Brown, & Finkbeiner, 2013; Mottet & Bootsma, 1999; Plamondon, 1995) and a *model-independent* framework (e.g., Freeman & Ambady, 1988; Meyer, Abrams, Kornblum, Wright, & Keith Smith, 2010). Although the model-dependent approaches have a long tradition in cognitive psychology (e.g., Wiener’s diffusion models, Rayleigh–Duffing model, minimum-jerk kinematics), the data-driven model-independent approaches are still appealing in terms of simplicity and feasibility as a considerable number of papers have also shown recently (Freeman 2016). In this paper, we will describe a novel mathematical procedure, called EMOT (Entropic MOuse Tracker), to analyze mouse-movement trajectories from a data-driven perspective. In particular, EMOT extends the main findings of the descriptive geometric approach (Freeman and Ambady 2010) in analyzing the basic features of the mouse trajectories (i.e., directions and amplitudes). However, unlike the latter approach, our proposal provides a more robust modelization of the trajectories in case of noisy and irregular movement paths and, in addition, it models a previously ignored component of the hand movement, motor pauses, which can reflect important aspects of the reaching process, for instance decisional uncertainty in classification tasks (Plamondon 1995; Johnson et al. 2012; Glaholt and Reingold 2011).

The reminder of this article is organized as follows. In the “Analyzing movement trajectories: a brief overview”, we provide a comprehensive overview of the perspectives used in analyzing mouse trajectories. In the “An entropic approach to mouse-tracking data”, we present our proposal together with its main characteristics. The “Two empirical applications” describes two case studies showing the application of the new proposal to two psychological datasets. The “Discussion” provides a discussion of the results of this study with comments and suggestions for future works whereas the “Conclusions” ends the article.

## Analyzing movement trajectories: a brief overview

To formally represent goal-directed movement trajectories, two main approaches can be recognized in the model-dependent framework. In the first one, movement paths are evaluated by fitting probabilistic models on the collected data (e.g., Wiener’s diffusion models, dynamic Markov chains) with the purpose of discovering important movement features, such as initiation process, decision phases, changes of direction (Friedman et al. 2013). Although this approach is quite successful in modeling mouse trajectories, in some circumstances (e.g., when the model assumptions cannot be completely satisfied) it might lead to problems of data representation (Voss and Voss 2008; White et al. 2010). In the second perspective, movement features like speed, acceleration, stability, and frictions are formally deducted using mathematical models (e.g., the Rayleigh–Duffing model - Mottet & Bootsma, 1999) and kinematic principles (e.g., minimum jerk, minimum energy - Engelbrecht, 1995; Flash & Henis, 1991; Plamondon, 2001). This approach has been widely adopted to study well-constrained class of movements usually recorded in simple tasks, as for instance smooth target-directed movements, planar movements, reaching with obstacles. However, to what extent it can account for highly structured and noisy movement paths is still under debate (Shadmehr 2005). Unlike the model-dependent approaches, the model-independent ones make use of a data-driven perspective to detect process features related to hand movements. In particular, they can be based either on some computational methods for movement decomposition such as, initiation time, pause time, and verification time (Walker et al. 1993; Meyer et al. 1988) or on basic descriptive geometric method to detect significant movement features, such as pointwise distances (Freeman and Ambady 2010; Hehman et al. 2015). Because our new proposal is mainly associated with this latter representation, in the following section we will briefly review the geometric approach.

### Descriptive geometric approach

DGA basically implements a simple data-analysis procedure. First, the raw computer-mouse trajectories are pre-processed by space-rescaling, time-normalization, and eventually averaging. Next, the ensuing trajectories are carved up by using various measures, such as maximum deviation (MD), area under the curve (AUC), *x-y* flips, sample entropy, and bimodal coefficient, which are all supposed to be good indicators of the ongoing dynamics underlying the cognitive tasks (see Fig. 1). These measures are finally used in post hoc analyses, like spatial attraction analysis, distributional analysis, and spatial disorder analysis (Hehman et al. 2015; Freeman and Ambady 2010).

However, despite its simplicity, DGA can suffer from a number of potentially relevant drawbacks. First, the time-normalization procedure can sometimes disregard important noise which may be instead signal (e.g., see: Lucero & Koenig, 2000; Tang & Müller, 2008). Second, in some circumstances, averaging multiple movement profiles may lead to inappropriate results, especially when empirical paths of movement show different amplitudes, phase variations, and curvatures (Ramsay 2006). Consequently, it is hard to choose a proper averaging operator (e.g., arithmetic, geometric, block-based) that preserves the profile information (Brown and Heathcote 2003). Third, due to the possible problem of multiple-maxima in the raw movement profile, the spatial attraction analysis may be biased. Indeed, as reaching movements are noisy, the recorded movement profiles may not completely show clear patterns with just one inflection point (this is the case of complex sinusoidal and quasi-cyclic trajectories). As a result, spatial measures like MD, AUC, and x/y flips may not be sensitive enough to recognize relevant subcomponents of the movement. Finally, the spatial measures cannot capture features or information associated with the presence of motor pauses during the motor execution. However, these pause patterns can potentially reveal important characteristics of the dynamic of a cognitive process such as, for example, decisional conflicts or uncertainty in categorization or choice tasks (Johnson et al. 2012; Glaholt and Reingold 2011; Plamondon and Alimi 1997).

In the next section, we will present an alternative mathematical method, EMOT, for movement trajectories that can overcome many of the drawbacks of DGA by providing a good compromise between model flexibility and model simplicity. In particular, EMOT extends many of the major conclusions from DGA (e.g., hand movements can provide real-time indicators of cognitive processing, dynamic competition during categorization can be capitalized by abrupt shift of movement profile) by using a *model-based representation* for the computer-mouse trajectories. In addition, this method will also provide a valuable way to detect other movement events (e.g., motor pauses) that may potentially reveal decisional uncertainty and/or conflicts during the cognitive task.

## An entropic approach to mouse-tracking data

*fast movements*, which correspond to rapid motor executions in the movement space (see Fig. 2a, points in green), and

*motor pauses*, occurring when the speed of the mouse cursor decreases and eventually breaks in a given region of the movement space (see Fig. 2a, points in the red ellipse. For further readings see: Hwang, Keates, Langdon, & Clarkson, 1988; Meyer et al., 2005).

^{1}By adopting an information-theoretic perspective, EMOT aims to quantify the decision process, which is codified by spatial components of computer-mouse trajectories, with a set of entropy measures within a single-trial approach. In particular, the noisy and dynamic

*x*-

*y*trajectory is initially mapped into a set of polar objects (distances and angles). Next, a linear histogram model of angles of movements is built, which stores the most important

*spatial*features of the computer-mouse trajectory such as location, directions, and amplitudes. Finally, the histogram model is used to determine a quantification of spatial events involved in the original movement path (e.g., competition between target and competing cues present on the screen) in terms of fast movements and motor pauses. The final step is realized according to a tailor-made entropy decomposition of the histogram model. In this way, similarly to DGA, our method will provide information concerning the spatial properties of a trajectory (i.e., location, direction, amplitude) with regards to both the target and competing cues. However, unlike DGA, EMOT supplies a model for the noisy mouse-movements and, in addition, a way to represent motor pauses that are usually neglected in the DGA approach. In fact, motor pauses may reveal important aspects of the dynamic cognitive processes such as, for example, decisional conflicts in categorization and/or decisional uncertainty associated with the participants’ behaviors in the experimental task (Johnson et al. 2012; Glaholt and Reingold 2011; Plamondon and Alimi 1997).

### Data representation and modeling

Let \(\mathbf {p} = (\mathbf {x}, \mathbf {y}) \in \mathbb {R}^{n} \times \mathbb {R}^{n}\) be an array of Cartesian coordinates associated to the streaming of *x-y* data of the computer mouse movements with *n* being the recorded number of movements points for a given participant in a specific trial. The modeling step consists of transforming the mouse movement trajectory into a spatial motion model, while retaining as much as possible of the properties of the original trajectory (i.e., location, direction, amplitude) present in the original movement space. Inspired by some previous works in the field of action recognition (Gowayyed et al. 2013; Kapsouras and Nikolaidis 2014; Chen et al. 2015) and topic analysis (Wang et al. 2013), we used the linear histogram of movement angles as a simple trajectory model (Shadmehr and Mussa-Ivaldi 2012; Calcagnì and Lombardi 2014). The use of a histogram model simplifies the study of trajectories and spatial features in a proper low-dimensional movement space. This facilitates, for instance, the emergence of relevant movement properties like the linear spatial attraction of the distractor on the target cue. More technically, let **p** ^{∗}=(**r**,* 𝜃*) be the transformed vector of

**p**, where \(\mathbf {r} = \sqrt {\mathbf {x}^{2} + \mathbf {y}^{2}}\) and

*=atan2{*

**𝜃****y**,

**x**}. In

**r**the power is element-wise whereas, for the sake of representation, the function atan2{.} is implemented using a special clockwise version which works in the subspace [0,3.14] (half quadrant). Next, the vector of angles \(\boldsymbol {\theta } \in \mathbb R_{+}^{n}\) is modeled by the histogram \(\mathcal {H}_{\theta } = (\boldsymbol {\eta }, \boldsymbol {\omega }) = \{(\eta _{1}, \omega _{1}),...,(\eta _{t}, \omega _{t}),...,(\eta _{T}, \omega _{T}) \}\) where

*η*

_{ t }is the

*t*

^{th}bin (

*t*=1,…,

*T*),

*ω*

_{ t }represents the frequency of occurrence of movement points in

*η*

_{ t }, and

*T*is the natural histogram parameter (Fisher 1995). The search for the best number of bins

*T*is an important point in the construction of the model \(\mathcal {H}_{\theta }\). As is known, several rational criteria (e.g., Sturges’ formula, Doane’s formula, Freedman–Diaconis’ rule) as well as automated procedures (e.g., see: Shimazaki & Shinomoto, 2007) can be employed to get appropriate estimates of

*T*. In our context, this search is automated using a dedicated trial-by-trial sensitivity analysis (Birgé and Rozenholc 2006).

^{2}

Figure 2 shows a graphical representation of the EMOT modeling step. In particular, panel (A) represents the prototypical case in which the competing cue C1 produces attraction. In this example, the support of the histogram spans (in angles) from C1 to C2 although the majority of frequencies is still located around C1. On the contrary, panel (C) reflects a situation in which mouse movements are largely located toward the final selected cue C2 while showing fewer directions toward C1 even though they span along the interval ranging from C1 to C2. This situation might occur when fast and hasty mouse movements do not relate with a real attraction for the competing cue C1. Finally, Panel (B) codifies a situation where both C1 and C2 cues produce a strong attraction on mouse movements with the consequence of an higher recorded uncertainty.

### Extracting motion features

*ϕ*(

*,*

**η***), which returns for each distinct bin*

**ω***η*

_{ t }the corresponding cumulative frequency \({\sum }_{s \leq t}\omega _{s}\) smoothed according to a standard interpolation algorithm (e.g., see: Eilers & Goeman, 2004). Next, the

*cumulative residual entropy*of

*ϕ*(

*,*

**η***) was computed as follows:*

**ω****1**is a

*T*×1 vector of all ones. The quantity

*ψ*is a modified version of the traditional Shannon entropy and has been firstly introduced by Rao et al. (2004). It is defined as a measure of the average amount of information and therefore is related with the degree of complexity in a given system (for further details see: Di Crescenzo & Longobardi, 2009; F. Wang & Vemuri, 2007).

^{3}In our context,

*ψ*is an overall measure of the

*total spatial motion information*captured by the movement model

*ϕ*(

*,*

**η***), which represents the amount of information reflected in the movement trajectory. In order to account for the sub-components of the trajectory, Eq. 1 is additively decomposed, as follows:*

**ω***ξ*and

*ζ*are the unknown residual cumulative entropies associated with the fast movement and the motor pauses, respectively. Similarly,

*and*

**τ***are*

**υ***T*×1 vectors of unknown weights of the models of fast movements and motor pauses, respectively. Furthermore, by letting

*ζ*=

*ζ*

_{1}+

*ζ*

_{2}to include the information related to competing and correct cues, Eq. 2 can be additionally decomposed as:

*ζ*

_{1}and

*ζ*

_{2}are the unknown cumulative residual entropies of motor pauses occurred in the direction of the competing cue and the target (see Fig. 2a), whereas

**υ**_{1}and

**υ**_{2}are

*T*×1 vectors of unknown weights of the corresponding components. The decomposition of the quantity

*ψ*allows for the identification of the movement components and their separate analysis. The overall estimation problem boils down in determining the values of the unknown quantities

*ξ*,

*ζ*

_{1}, and

*ζ*

_{2}that satisfy the equality (3), given the observed data

*.*

**𝜃**#### Estimating *ξ*, *ζ* _{1}, *ζ* _{2}

In line with the descriptive nature of our proposal and in order to retain the single-trial approach upon which EMOT is based, we estimated *ξ*, *ζ* _{1}, *ζ* _{2} by reformulating the decomposition as a non-linear optimization problem (NLP) within the minimum Kullback–Leibler framework (Ciavolino and Calcagnì 2015; Good 1963), a dual version of the well-known Jayne’s maximum entropy principle (Kapur and Kesavan 1992). However, in order to apply this procedure, we first need to define a set of external feasible information about *ξ*, *ζ* _{1}, and *ζ* _{2} that will constrain the class of possible solutions of our decomposition problem.

*to derive some*

**𝜃***proxies*that can be used in solving our problem. In particular, let \(\boldsymbol {\theta }_{0} = \mathcal {F}_{0}(\boldsymbol \theta )\) be the

*zeroth-frequency moment*of

*where*

**𝜃**

**𝜃**_{0}contains only distinct element of

*with no ties whereas \(\mathcal {F}_{0}\) is a standard algorithm for this purpose (Coppersmith and Kumar 2004).*

**𝜃**^{4}Like for \(\mathcal {H}_{\theta }\), we can also define the histogram for

**𝜃**_{0}. More specifically, let \(\mathcal {H}_{\theta _{0}} = (\boldsymbol \eta , \boldsymbol \pi ) = \{(\eta _{1},\pi _{1}),...,(\eta _{t},\pi _{t}),...,(\eta _{T},\pi _{T})\}\) be the histogram model for

**𝜃**_{0}, where

*represents the occurrences of the movements (without duplicates) in*

**π**

**𝜃**_{0}. Given its nature,

*can serve as a proxy for*

**π***. By contrast, to define feasible proxies for*

**τ**

**υ**_{1}and

**υ**_{2}, we will make use of a procedure similar to the one adopted in the “spatial pyramid matching”, a pattern recognition technique which is used to recursively divide an image histogram into sub-histograms of features (Lazebnik et al. 2009). More precisely, let us define:

*η*

_{ S }of \(\mathcal {U}\). Next, we set:

**λ**_{1}and

**λ**_{2}can be used as proxies for the quantities

**υ**_{1}and

**υ**_{2}.

*,*

**τ**

**υ**_{1}, and

**υ**_{2}are the associated

*T*×1 unknown vectors.

*,*

**π**

**λ**_{1}and

**λ**_{2}, the NLP problem can be set as follows:

**1**is a vector of ones of appropriate order whereas the objective function to be minimized is the well-known Kullback–Leibler (KL) divergence, which is expressed as a function of the unknowns quantities

*,*

**τ**

**υ**_{1},

**υ**_{2}and the proxies

*,*

**π**

**λ**_{1},

**λ**_{2}. Note that the first constraint in the equation ensures that all the estimates are still compatible with the decomposition of the quantity

*ψ*whereas the other constraints are just normalization factors which guarantee well-posed estimates.

Considering the nature of the problem (5), the (approximated) solutions can be found numerically using an interior-point method that allows for efficient and effective handling of the equality constraints described in Eq. 4 (Gill et al. 1981). A preliminary pilot study confirmed this expectation by showing that the algorithm was indeed fast and accurate enough to provide stable and convergent numerical solutions.

### Implementation

The EMOT method is implemented as an add-on package for the Matlab environment and includes function for data modeling, estimating parameters, and performing graphical analyses of trajectories and extracted measures. All the features involved in the analysis (e.g., NLP algorithm and parameters, histogram parameters, sensitivity analysis) can be set by the user. Moreover, different types of analysis (e.g., subject-by-subject and global) with different characteristics can be selected. Finally, the application provides a single output containing all the results of the analysis. The EMOT tool works with the raw *x-y* computer-mouse trajectories that can be collected with any recording software. The EMOT package is freely available at http://polorovereto.unitn.it/~antonio.calcagni/emot.html.

### Interpretation of the EMOT measures

### Measure

*ψ* The cumulative residual entropy is used as an overall quantity of the spatial motion information expressed by the movement data. As such, *ψ* conveys the overall dynamics of the *x*-*y* trajectory executed in the movement space (e.g., degree of freedom of the movement, spatial exploration) and correlates to the amount of spatial uncertainty due to the movement dynamics. Consequently, it takes its minimum value when the movement is simply executed in the direction of the target C2, i.e., when the movement shows no attraction toward the competing cue C1, whereas takes its maximum value when the movement dynamics show attraction/competition toward the competing cue C1 (see Fig. 2a). This interesting property makes *ψ* directly comparable with MD/AUC measures of the DGA approach. However, since *ψ* is calculated using the models \(\mathcal {H}_{\theta }\) and *ϕ*(* η*,

*) rather than directly*

**ω***, it is more robust to noisy and uneven movement trajectories than MD/AUC.*

**𝜃**### Decomposition of

*ψ* The additive decomposition, *ψ* = *ξ* + *ζ* _{1} + *ζ* _{2}, allows for a simple and effective analysis of the motor information and provides insights into the mouse trajectories recorded during the task. Indeed, the measures *ξ*, *ζ* _{1}, and *ζ* _{2} quantify motion events that are relevant in the mouse-tracking context, as they might reveal important aspects of the involved cognitive processes, such as decisional conflicts and decisional uncertainty (Johnson et al. 2012; Glaholt and Reingold 2011). In particular, the component *ξ* quantifies rapid movement events associated, for instance, with motor-execution behaviors, which can arise after a decision has been made. Instead, the components *ζ* _{1} and *ζ* _{2} represent pause movement events that are associated, for example, with processes of goal formulation and/or reformulation and can uncover the dynamics of the decision process (Johnson et al. 2012; Glaholt and Reingold 2011).

## Two empirical applications

In this section, we will shortly present two applications of the EMOT approach based on two already published datasets. In the first application, we will describe a single-trial analysis on data coming from a standard categorization task (Dale et al. 2006). Here, we will evaluate (a) how the entropy-based indices behave and (b) how they can be interpreted when empirical movement profiles are considered individually. In the second application, we will also evaluate the EMOT measures on a group analysis of lexical decision data (Barca and Pezzulo 2012). In both the applications our measures will be also matched to a set of the most frequently used DGA measures. Note that the applications we report here have only illustrative purposes with the aim to highlight the importance of considering both the movement and pause measures as relevant indexes of cognitive processing.

### Categorizing atypical exemplars

### General context and motivation

Categorizing stimuli with atypical nature usually involves competition and uncertainty in participants that have to choose among categories with high similarities. Indeed, typical stimuli are also recognized faster and more accurately than atypical ones (Dale et al. 2007; Rips et al. 1973). By using the mouse tracking methodology, Dale et al. (2006) have investigated the effect of stimulus typicality in the context of the categorization of natural stimuli (e.g., animals). In particular, they proposed a simple semantic categorization task where basic level animal names (e.g., *gorilla*) were assigned to their respective superordinate category (e.g., *mammal*). Participants were asked to categorize animal stimuli in the correct category by choosing between two possible assignments (e.g., *gorilla*: *mammal* vs. *reptile*) with the hypothesis that stimuli showing higher proximity with the incorrect category will result in larger mouse trajectories reflecting competition and uncertainty in the categorization process. We expect that task competition and uncertainty would be reflected both in terms of fast movements and motor pauses.

### Data, measures, procedure

The sample is represented by 34 right-handed students from the Cornell University. Lexical stimuli (125 names of animals) belonging to the natural superordinate categories of mammal, fish, reptile, bird, and insect, were extracted from the McRae et. al.’s study (McRae et al. 1997) together with their feature-semantic measures and measures of similarities. Participants were presented with two category names (target and competitor) placed in the right and left-hand corner of a computer screen whereas the stimulus name was placed on the bottom of the screen. Participants were asked to categorize the animals by clicking the computer-mouse on their correct categories. In the meantime, the x-y streaming of the mouse coordinates were recorded. The positions of target and competitor and, for each trial, the competitor category were randomly chosen among the four possible animal categories.

### Data analysis and results

We re-analyzed the original *x-y* mouse trajectories data using the EMOT approach. In addition, we also computed some well-known DGA measures (MD, AUC, xFlips, samEnt: sample entropy, AC: acceleration components) using the package mousetrap developed by P. J. Kieslich and F. Henninger.^{5} The AC measure was instead calculated as indicated in Duran et al. (2010).

*ψ*=

*ξ*=2.37) with no motor pauses in the direction of the competing lexical category

*reptile*. In this example, EMOT and DGA measures convey converging results.

*seal*in the correct category

*mammal*because of the lexical competition of the other category

*fish*. This is reflected in the trajectory as well as in the pauses occurring toward the competing category. The empirical movement path is defined by two main features: a first rapid movement toward the incorrect category, which ends with a motor pause and a second rapid movement toward the correct category. The overall dynamics of the movement is largely due to the two fast movements (

*ξ*=2.94, 91 %) with only a motor pause for the competing category (

*ζ*

_{1}=0.28, 9 %;

*ζ*

_{2}=0, 0 %). This profile is an example of a movement path composed by two discrete sub-movements separated by a pause (e.g., see: Hwang et al., 2005; van der Wel, Eder, Mitchel, Walsh, & Rosenbaum, 2009), which is difficult to reconcile with the assumption of continuous movement trajectories assumed by some of the DGA interpretations (e.g., see: J. Freeman, Dale, & Farmer, 2011).

*fish*. Here, the pattern shows a movement composed by three sub-movements with three areas of motor pause in the direction of both categories. The movement starts with a rapid ballistic trajectory toward the correct target

*mammal*, turns toward the incorrect category

*fish*, and turns again toward the correct

*mammal*where it definitely stops. In this example, all the three of the EMOT measures were activated (

*ξ*=1.65, 75 %;

*ζ*

_{1}=0.35, 16 %;

*ζ*

_{1}=0.18, 9 %). Note that for this pattern the DGA measures do not capture all the richness capitalized by the movement trajectory. For instance, AUC = 0 suggests that no attraction has been exercised by the competing category

*fish*.

*mammal*followed by a motor pause just before the final clicking. The overall movement dynamic is mainly due to fast motor executions (

*ξ*=1.56, 81 %) and a motor pause in the area of the target (

*ζ*

_{2}=0.36, 19 %;

*ζ*

_{1}=0, 0 %). Note that in this case some of the DGA measures, such as MD and AUC, show negative values, which are hard to interpret in substantive terms. Moreover, xFlips suggests a kind of complexity along the

*x*-axis although there is no displacement toward the category

*bird*. Similarly, Fig. 7 depicts a movement path characterized by a sequence of submovements along the direction of the correct category

*mammal*, conjoined by two pauses. The overall dynamics of the movement is mainly due to movement executions (

*ξ*=0.78, 82 %). Motor pauses, which are in the direction of the target (

*ζ*

_{2}=0.13, 18 %;

*ζ*

_{1}=0, 0 %), may reflect here the activation of verification components (e.g., see: Meyer et al., 1988).

### A lexical decision task

In this second application, we studied the behavior of EMOT and DGA measures by looking at participants’ performance in a lexical decision task.

### General context and motivation

Lexical decision is one of the most used task in psycholinguistic literature since the 1960s. Generally, in this task, participants see a printed stimulus on the screen (e.g., *hand*) and have to perform a dichotomous choice to decide whether it is a real word or not. The lexical decision task is widely used in the study of visual world recognition and reading. Since this task is very simple and versatile, it has been employed in tens of different experimental situations and is suitable for studying multiple levels of the linguistic processing (e.g., Hawkins, Brown, Steyvers, & Wagenmakers, 2012; Norris & Kinoshita, 2008; Yap, Balota, Tse, & Besner, 2008).

In this second application, we assessed the extent to which the overall entropy measure *ψ* and each of its subcomponents, *ξ*, *ζ* _{1}, and *ζ* _{2}, are affected by a stimulus type factor composed by words, pseudowords (i.e., sequences of letters that do not exist in the language but can be read), and random strings (i.e., random sequence of letters that are phonotactically illegal in the language) in a lexical decision task. We expect that the entropy measure *ψ* will be characterized by a pattern similar to that of the geometric measure MD/AUC with in general pseudowords being associated with larger levels of uncertainty in the decision process. Moreover, we also assessed the importance played by the pauses or pauses (measured by means of the quantities *ζ* _{1} and *ζ* _{2}) in shaping the level of uncertainty associated with more difficult stimuli.

### Data, measures, procedure

As a case study, we take advantage of the work by Barca and Pezzulo (2012) that is, for our purpose, a perfect case of lexical decision. They ran a lexical decision experiment in Italian by adopting a stimulus type factor with four different categories: Words of high written frequency (HF, e.g., *acqua* ’water’), words of low written frequency (LF, e.g., *cervo* ’deer’), pseudowords (PW, e.g., *dorto*), and strings (STR, e.g., *btfpr*). Participants saw a total of 96 stimuli (1/4 for each category), once at the time, and had to categorize each stimulus as either a word or a nonword; the two alternative answers were displayed at the top left and top right of the screen and the choice was performed by reaching one of the two alternatives with the mouse.

### Data analysis and results

We re-analyzed the original *x-y* mouse trajectories collected by Barca and Pezzulo (2012) by using the EMOT approach and compared the entropy measures with the standard measures of the DGA (i.e., MD, AUC, xFlips, samEnt, and AC) that are computed as described for the first application. Moreover, in our analysis, other than using the stimulus type factor used by Barca and Pezzulo (2012), we added a further psycholinguistic variable, bigram frequency, as a covariate. Bigram frequency is the frequency with which adjacent pairs of letters (bigrams) occur in printed texts; for its characteristics, it may be considered as a measure of orthographic typicality (see, e.g.,: Hauk, Davis, Ford, Pulvermüller, & Marslen-Wilson, 2006) and we used it as a covariate in the analysis to further reduce error variance. Only bigram frequency was used since it was the only psycholinguistic variable that could be computed for all the stimuli.

#### Examples of single-case analysis

*NW*before reversing in the direction of

*W*. This pattern suggests a prolonged decision uncertainty rather than a continuous competition between the two categories. The overall movement dynamics is proportionally due to motor executions (

*ξ*=2.37, 62 %) and pauses (

*ζ*

_{1}=1.08, 28 %;

*ζ*

_{2}=0.34, 10 %). Figure 9 shows instead a quasi-cyclic trajectory involving several discrete sub-movements and pauses with no attraction in the direction of

*NW*. The overall movement dynamics is mainly ascribed to motor executions (

*ξ*=2.21, 79 %) and pauses toward the target (

*ζ*

_{2}=0.58, 21 %;

*ζ*

_{1}=0, 0 %). Interestingly, in this case, DGA measures fail to recognize the characteristics of this profile: AUC is close to zero because of the quasi-cyclic trajectory shown by this profile, which yields summing positive and negative portions of area at the same time.

*ξ*=1.00, 57 %) whereas it is interesting to note that motor pauses are approximately the same in their values (

*ζ*

_{1}=0.379, 22 %;

*ζ*

_{2}=0.381, 21 %). This suggests that the participant showed greater uncertainty in categorizing the stimulus as nonword or word. Indeed, he/she first hesitated toward

*W*, then toward

*NW*, before selecting the correct final choice. In sum, all the three examples illustrated in this second application would result in some level of interpretation difficulty for the basic DGA measures (particularly for MD and AUC).

#### Group analysis

*ψ*and MD (and to some less degree between

*ψ*and AUC), whereas only mild correlations were observed between

*ψ*and the other DGA measures (xFlips, samEnt, and AC). Because of the model definition, we naturally observed also positive correlations between

*ψ*and its subcomponents,

*ξ*,

*ζ*

_{1}, and

*ζ*

_{2}, with

*ξ*showing the largest correlation among the three terms. Interestingly, the correlational analysis showed a negative association between the two pause entropies

*ζ*

_{1}and

*ζ*

_{2}. Finally, the last line of Table 1 reports the nonparametric correlations between the overall response times (RTs) and the mouse tracking measures. From a quick inspection of these correlations, it seems clear that the mouse tracking measures and RTs are independent quantities characterizing different components of the process. In particular, the overall duration of a lexical decision task does not capture the motor uncertainties unfolding during the decisional process.

Spearman’s correlations for all possible distinct pairs of measures

| | | | MD | AUC | xFlips | samEnt | AC | |
---|---|---|---|---|---|---|---|---|---|

| − | − | − | − | − | − | − | − | − |

| 0.779 | − | − | − | − | − | − | − | − |

| 0.315 | 0.385 | − | − | − | − | − | − | − |

| 0.372 | -0.118 | -0.378 | − | − | − | − | − | − |

MD | 0.902 | 0.772 | 0.336 | 0.258 | − | − | − | − | − |

AUC | 0.740 | 0.634 | 0.109 | 0.353 | 0.731 | − | − | − | − |

xFlips | 0.337 | 0.231 | 0.187 | 0.103 | 0.364 | 0.269 | − | − | − |

samEnt | 0.419 | 0.359 | 0.242 | 0.078 | 0.443 | 0.296 | 0.233 | − | − |

AC | 0.447 | 0.350 | 0.169 | 0.159 | 0.426 | 0.348 | 0.570 | 0.406 | − |

RT | 0.072 | 0.074 | −0.039 | 0.033 | 0.096 | −0.036 | −0.178 | 0.085 | −0.068 |

*ψ*measure (as dependent variable), we ran a robust linear mixed model

^{6}(RobLMM) using stimulus type as factor (with HF as reference level of the factor) and bigram frequency as covariate; random intercepts for subjects and items were also included in the model. Table 2 reports some descriptive statistics of the subcomponents of

*ψ*as a function of the four levels of the experimental factor. To compare the results for

*ψ*, we also run similar analyses using the DGA measures as new dependent variables. Table 3 reports the overall results of the RobLMM analysis. As expected, the results of the robust regressions showed very similar patterns for AUC, MD, and

*ψ*: In all the models, there was a significant difference between PW and HF (AUC:

*β*

_{pw}=0.123, SE = 0.016,

*t*=7.655; MD:

*β*

_{pw}=0.233, SE = 0.018,

*t*=13.212;

*ψ*:

*β*

_{pw}=10.574, SE = 0.832,

*t*=12.714) with an increasing degree of uncertainty in PW with respect to HF. However,

*ψ*showed also marginal effects for STR (

*β*

_{str}=2.231, SE = 1.117,

*t*=1.998). No further effect reached significance in the three models. By contrast, sample entropy and acceleration showed a moderate effect between LF and HF (samEnt:

*β*

_{lf}=0.012, SE = 0.006,

*t*=2.034; AC:

*β*

_{lf}=0.015, SE = 0.006,

*t*=2.488) but only sample entropy was significantly different between PW and HF (samEnt:

*β*

_{pw}=0.015, SE = 0.006,

*t*=2.488). The last DGA measure, xFlips, did not show any significant difference. Finally, the results for the RT analysis showed a significant difference between PW and HF (

*β*

_{pw}=162.011, SE = 9.575,

*t*=16.919) as well as a significant difference between LF and HF (

*β*

_{lf}=41.076, SE = 9.157,

*t*=4.486). In particular, this latter result about low frequency words showed how response time and some of the mouse trajectory measures, such as the EMOT measures, MD and AUC, captured different facets of the lexical decision process.

Mean values of the entropy, geometric, and response time measures as a function of the experimental factor

| | | | RT | AUC | MD | xFlips | samEnt | AC | |
---|---|---|---|---|---|---|---|---|---|---|

HF | 25.810 | 21.278 | 0.771 | 3.783 | 1421.582 | 0.767 | 0.286 | 8.201 | 0.199 | 14.659 |

(14.334) | (13.391) | (1.729) | (5.156) | (180.971) | (0.310) | (0.353) | (4.395) | (0.130) | (6.591) | |

LF | 27.509 | 22.580 | 0.779 | 4.174 | 1463.200 | 0.801 | 0.335 | 7.882 | 0.215 | 14.599 |

(14.739) | (14.263) | (1.735) | (5.286) | (201.099) | (0.336) | (0.410) | (4.181) | (0.155) | (6.316) | |

PW | 36.928 | 31.699 | 1.472 | 3.834 | 1578.814 | 0.912 | 0.573 | 8.437 | 0.212 | 15.186 |

(19.088) | (18.143) | (2.451) | (5.064) | (181.363) | (0.392) | (0.531) | (4.165) | (0.131) | (6.130) | |

STR | 26.272 | 21.994 | 0.969 | 3.318 | 1422.084 | 0.739 | 0.292 | 8.579 | 0.198 | 14.581 |

(14.402) | (13.243) | (1.773) | (4.798) | (173.742) | (0.316) | (0.350) | (4.442) | (0.133) | (6.126) |

Summary of the main effects observed on the different measures

| | | | RT | AUC | MD | xFlips | samEnt | AC | |
---|---|---|---|---|---|---|---|---|---|---|

HF - LF | n.s. | n.s. | n.s. | n.s. | ** | n.s. | n.s. | n.s. | * | n.s. |

HF - PW | ** | ** | ** | n.s. | ** | ** | ** | n.s. | * | * |

HF - STR | * | n.s. | * | n.s. | n.s. | n.s. | n.s. | n.s. | n.s. | n.s. |

After having evaluated the overall entropy effect as measured by *ψ*, we further investigated the single components, *ξ*,*ζ* _{1}, and *ζ* _{2}, of the EMOT model by running different RobLMMs on the same set of independent variables. About the entropy measure of fast movements, *ξ*, the result of the RobLMM analysis showed a significant difference between PW and HF (*β* _{pw}=9.625, SE = 0.772, *t*=12.461), which was also the only observed statistically significant result in the model. The pattern was similar to that of the overall measure *ψ*, with a larger degree of uncertainty in PW as compared to HF. For the pauses toward the distractor, *ζ* _{1}, the results of the robust analysis showed a significant effect for both PW and STR (*β* _{pw}=0.282, SE = 0.054, *t*=5.222; *β* _{str}=0.207, SE = 0.073, *t*=2.856) with larger levels of uncertainty in PW and STR with respect to HF. By contrast, the pauses toward the target (*ζ* _{2}) did not show any significant effect in the RobLMM analysis. Overall, the pattern of results showed that stimulus type affected two of the three components of the overall entropy measure *ψ*. In particular, the level of entropy is larger for PW than HF both in pure movements and pauses toward the distractor (*ξ* and *ζ* _{1}); by contrast, stimulus type does not affect the level of entropy for pauses toward the target, *ζ* _{2}. These results indicate that in some psycholinguistic tasks, such as lexical decision with words, pseudowords, and random strings, pure movements play a relevant role in characterizing the decisional process, with in general easier stimuli being associated with lower levels of movement entropy as measured by mouse trajectories. However, the uncertainty related with pure movement does not tell the entire story, as also the entropy associated with pauses, in particular that occurred in the direction of the distractor, is an important aspect in describing the mechanisms involved in the decisional task. As far as we know, this is the first time that motor pauses have been modeled and investigated to better understand the cognitive processes involved in such empirical paradigms.

## Discussion

The new EMOT approach aimed to analyze computer-mouse trajectories recorded during mouse-tracking experiments, in terms of their constitutive components (fast movements and motor pauses). EMOT offers a way not only to detect such components, but also to quantify them in terms of entropy, a means that helps to measure decisional uncertainty during two-choice categorization tasks (see Application 1). This is an important feature of EMOT, which is different from other existing approaches, such as DGA (Freeman and Ambady 2010) or parsing based techniques (Hwang et al. 2005).

In general, the decomposition of *ψ* entails a set of relations among the subcomponents *ξ*, *ζ* _{1}, and *ζ* _{2} that can guide a researcher to interpret EMOT results. The first component *ξ* is related to (rapid) movements. Because two-choice categorization tasks naturally require a final decision, a *good* sampled trajectory should always show positive values of *ξ*, which indicates that a dynamic movement occurred. Even very fast movements always involve a number of motor executions that are detected by the subcomponent *ξ*. Consequently, movement profiles with *ξ*=0 would reveal problems during sampling of computer-mouse movements. Given its nature, *ξ* is similar to the more traditional DGA measures and may be associated to both the decisional unfolding and fulfillment in categorization tasks (e.g., see: Freeman & Ambady, 2010).

With regards to *ζ*, positive values are not mandatory events, as indicated by those profiles showing rapid motor execution without relevant slowing-down phenomena (e.g., see Fig. 3). This could be the case, for example, when the decision process either has been completed before a participant moves the pointer or the decision unfolds smoothly. However, the most typical profiles involve positive values for *ξ* followed by positive values for one of the two components *ζ* _{1} or *ζ* _{2}. We suggest that *ζ* _{1} and *ζ* _{2} mirror different processes. A prominence of *ζ* _{1} may be associated to a conflict between the target and the competitor category, with larges values denoting a stronger competition exerted by the distractor (Fig. 4 exemplifies this process). Differently, a prominence toward *ζ* _{2} seems to be associated to the verification component of the decision process, which may be tentatively linked to the activity of the monitoring system (e.g., see: Resulaj, Kiani, Wolpert, & Shadlen, 2009).

In sum, the EMOT approach as well as its empirical validation give us the chance to make a more general consideration on the dual-choice task. As nicely exemplifies Fig. 5, the categorization process does not necessarily involve a continuous mouse attraction toward the distractor; instead, in some cases, the process seem to unfolds through discrete consecutive movements (e.g., see: Fishbach, Roy, Bastianen, Miller, & Houk, 2007).

Exemplary relations among *ξ*, *ζ* _{1}, *ζ* _{2} and their interpretations

| Movement dynamics is mainly represented by fast motor executions whereas entropy of pauses toward distractor is higher than that for the target. This is a typical case with increasing levels of cognitive processing due to the distractor. The special case |

| This is a typical case with a slight increasing of cognitive processing due to the distractor. The overall dynamics is mainly due to motor executions and pauses in the direction of the target. This indicates a decision processing mainly oriented toward the target category, with a mild tendency to a verification component of the final decision. Examples of these patterns are represented in Figs. 6, 7, 9. |

| This pattern codifies particular situations where an increasing in cognitive processing is caused by the distractor: this increase appears as a large amount of pauses instead of a typical dynamic continuous movement. Figure 5 shows an example of this pattern. |

| This pattern regards two different configurations: (i) |

### Potential and limits of the EMOT approach

Although temporal analyses might shed light on the dynamics of the cognitive characteristics of the movement (Jonsen et al. 2005), we recognize that the EMOT framework uses the spatial components of empirical trajectories as the only informational source. However, this simplification can be easily justified. First, we were interested in analyzing spatial descriptors of the movement, which are important in the two-choice categorization domain (like *cue attraction*). This entailed the use of a trajectory-oriented approach of analysis, focusing on the extraction of specific location parameters (e.g., absolute or relative angles) used in subsequent statistical analyses (Demšar and Virrantaus 2010). Second, as also pointed out by the results of our second application, the analysis of the spatial information provided different insights into the mechanisms underlying mouse-tracking tasks, which are not always uncovered by the temporal information (Georgopoulos et al. 1981). Moreover, the use of more complex models considering both spatial and temporal information together, such as the correlated random walk (CRW), usually requires some additional assumptions regarding the movement generation phase and the type of movement structure (e.g., navigation with no references, movement patters with no repetition. See: Bergman, Schaefer, & Luttich, 2000; McClintock et al., 2012), which would be difficult to test in the mouse-tracking context. By contrast, EMOT does not require specific assumptions as it provides a descriptive/phenomenological representation of the process. Indeed, it offers a simple mathematical method to summarize the movement trajectories with a small set of spatial descriptors and measures.

Like for all mouse-tracking methods, also the quality of EMOT results are strictly related to those of the original *x*-*y* data registration. Although computer-mice have notable advantages (e.g., low cost, easy availability), they have limitations that may cause problems to the kinematic properties of the trajectories (O’Really and Plamondon 2011). Because some setting variables, such as mouse gain, handedness, arm positioning, can affect results of mouse-tracking analyses, computer-mouse devices should be carefully used and possibly calibrated before running experimental tasks (Flodgren et al. 2007; O’Really and Plamondon 2011). For instance, in a two-choice categorization task, as those discussed in this paper, mouse gain may affect the reaching trajectory of the mouse by intensifying or diminishing the level of smoothness of the trajectory. For example, cursor speed may deteriorate the fine-grained movements of the mice. Consequently, eventual competition/curvature of the trajectories could result in artifacts arising from the lack of precision of the hand. Indeed, as is known, larger movement amplitudes are usually required by low mouse gain whereas a higher precision is instead needed with high mouse gain (Dillen et al. 2005). Other factors such as cues direction, inter-distances among cues, size/width of the screen should be carefully checked since they may affect the quality of computer-mouse recordings (Phillips and Triggs 2001). Also, the age of participants should be considered before performing a computer-mouse task because aging may seriously affect motor control of the hand by reducing precision and smoothness of the movement (Smith et al. 1999). Consequently, also EMOT results may be influenced by kinematic characteristics, especially for the quantification of fast movements and pauses. However, since EMOT only works with *x*-*y* *spatial information*, kinematic variables that may affect the temporal resolution of the data (e.g., gain, acceleration) do not largely affect its measures. With this respect, a first simulation study conducted by varying acceleration, mouse gain, and direction of movements confirmed such intuition (see Supplementary materials). Anyway, as for other mouse-tracking based analytic techniques, setting computer-mouse devices in an appropriate way would certainly improve the general performances of the EMOT representation and reduce the effect of artifacts on the quantification of movement features such as pauses.

Despite some potential negative impact of uncontrolled kinematic variables, several advantages can be still advocated for the EMOT approach. For instance, comparing with DGA, it extracts movement features using a trial-by-trial approach and, consequently, the spatial measures are computed using all the information available in the empirical trajectories as they are recorded. An additional advantage of our method concerns the comparison of different movement profiles. For example, when researchers want to analyze whether two or more movement paths differ from one another in terms of spatial information. While this is impractical for other data-driven approaches, in our proposal different profiles can be efficiently compared in terms of *cumulative Kullback-Leibler entropy* (Baratpour and Rad 2012). For instance, given two movement profiles *ϕ* _{1} and *ϕ* _{2}, which are associated with two movement data **𝜃**_{1} and **𝜃**_{2}, their divergence can be calculated as *χ*(*ϕ* _{1},*ϕ* _{2})=[* δ*− log(

*)−*

**δ****1**]

^{ T }(

**1**−

*ϕ*

_{1}), where the vector

*is obtained from the element-wise ratio between the vectors*

**δ****1**−

*ϕ*

_{1}and

**1**−

*ϕ*

_{2}(with

**1**being a vector of ones of appropriate order). This measure reaches its minimum value (

*χ*=0) when the movement profiles

*ϕ*

_{1}and

*ϕ*

_{2}are the same, that is to say when the two movement paths provide the same quantity of spatial information.

### Further extensions

As for any method, EMOT might be improved in various aspects. For example, the extension of our approach to multi-choice contexts, where participants have to assign stimuli to several possible categories. In a similar way, EMOT could be modified to handle with the problem of multi-trial integration, where entropy measures would need to be extracted from the distributions of the participants’ trials. This latter would turn out to the modelization of the error components across trials and participants, which would require the use of a stochastic decomposition of the equality *ψ* = *ξ* + *ζ* _{1} + *ζ* _{2} instead of the deterministic representation used throughout the paper. All these points need to be considered in further investigations.

## Conclusions

In this paper, we described a novel approach to analyze spatial data arising from mouse-tracking experiments and offer a way to extract spatial descriptors of the movement in the context of two-choice categorization tasks. In particular, our contribution extended the main findings of the descriptive geometric approach (DGA) by providing a way to decompose the spatial information provided by the movement trajectories. We argued that mouse movement trajectories can be regarded as composed of fast movement executions and motor pauses, which are both important for a full understanding of the processes underlying a mouse-tracking task. To illustrate the utility of the new method, we applied our new approach to two real datasets of mouse-tracking data, representing typical categorization tasks in a two-choice paradigm. As expected, we were able to identify interesting spatial movement features, describing characteristics of the processes involved in this type of tasks, such as the decisional uncertainty. Moreover, the second application showed how the proposed entropy measures can capture facets of the categorization process, which would not be represented using standard RT as well as other geometric or distance-based measures. Overall, these results highlighted how differences in terms of global movement trajectories can be better explained considering their constituent components (i.e., fast movement and pauses).

In conclusion, this study has yielded valuable findings in the decomposition of movement trajectories in terms of fast movements and pauses. Indeed, the information related with the sole movement does not tell the entire story, as also pauses, in particular those occurring in the direction of the competing cue, are important aspects in describing the mechanisms involved in decisional tasks. As far as we know, this is the first time that motor pauses have been modeled and investigated to better understand the cognitive processes involved in mouse-tracking studies. We believe that new research exploring and analyzing movement trajectories will benefit from the advantages gained by the EMOT method.

## Footnotes

- 1.
Technically, since trajectories are represented as collections of Cartesian coordinates points ordered in the natural way they were recorded (the temporal information is disregarded in spatial analyses), motor pauses are conceived as those parts of a trajectory with equal—or almost equal, in a given small neighborhood—Cartesian coordinates, whereas fast movements are those parts with different Cartesian coordinates that remain when motor pauses are removed from the original trajectory (e.g., see: Hogan & Sternad, 2007).

- 2.
In the EMOT context, the best value for

*T*is sought adopting an iterative single-trial sensitivity analysis. Particularly, the algorithm iteratively evaluates stability and fluctuations of the EMOT results over a set of admissible*T*’s. As for multi-objective optimization techniques, the best*T*is determined with a robust integration of sub-optimal*T*’s. Finally, the goodness of the solution is measured with an*accuracy index*which relates on how reliable is the final result. Generally, low values for such index are caused by a low quality of the movement profile (e.g., bad*x*-*y*data sampling, sparse data). - 3.
Although several measures of information are available in this context (e.g., Shannon entropy, fractal entropy) we opted for the cumulative residual entropy because: (a) it can be naturally applied on objects like

*ϕ*(,**η**), (b) it is defined in both discrete and continuous domains, and (c) it is applicable to both bimodal and unimodal models of \(\mathcal {H}_{\theta }\) (Rao et al. 2004).**ω** - 4.
As example, consider the array of angles

=(0.23,0.23, 0.45, 0.88, 0.88, 0.88, 1.10, 1.10). Extracting its zeroth-frequency moment leads to the vector**𝜃****𝜃**_{0}=(0.23,0.45,0.88,1.10), which contains only the unique elements of.**𝜃** - 5.
- 6.
To run the analyses, we used the R package robustlmm by Manuel Koller. The package is freely available on the CRAN repository.

## Notes

### Acknowledgments

The authors thank Dr. Rick Dale and Dr. Laura Barca for providing the datasets used in the Application 1 and in the Application 2, respectively.

## Supplementary material

## References

- Baratpour, S., & Rad, A. H. (2012). Testing goodness-of-fit for exponential distribution based on cumulative residual entropy.
*Communications in Statistics-Theory and Methods*,*41*(8), 1387–1396.CrossRefGoogle Scholar - Barca, L., & Pezzulo, G. (2012). Unfolding visual lexical decision in time.
*PloS One*,*7*(4), e35932.CrossRefPubMedPubMedCentralGoogle Scholar - Bergman, C. M., Schaefer, J. A., & Luttich, S. (2000). Caribou movement as a correlated random walk.
*Oecologia*,*123*(3), 364–374.CrossRefPubMedGoogle Scholar - Birgé, L., & Rozenholc, Y. (2006). How many bins should be put in a regular histogram.
*ESAIM: Probability and Statistics*,*10*, 24–45.CrossRefGoogle Scholar - Brown, S., & Heathcote, A. (2003). Averaging learning curves across and within participants.
*Behavior Research Methods, Instruments, & Computers*,*35*(1), 11–21.CrossRefGoogle Scholar - Calcagnì, A., & Lombardi, L. (2014). Dynamic fuzzy rating tracker (DYFRAT): A novel methodology for modeling real-time dynamic cognitive processes in rating scales.
*Applied Soft Computing*,*24*, 948–961.CrossRefGoogle Scholar - Chen, G., Clarke, D., Giuliani, M., Gaschler, A., & Knoll, A. (2015). Combining unsupervised learning and discrimination for 3D action recognition.
*Signal Processing*,*110*, 67–81.CrossRefGoogle Scholar - Ciavolino, E, & Calcagnì, A (2015). Generalized cross entropy method for analysing the SERVQUAL model.
*Journal of Applied Statistics*,*42*(3), 520–534.CrossRefGoogle Scholar - Coco, M. I., & Duran, N. D. (2016). When expectancies collide: Action dynamics reveal the interaction between stimulus plausibility and congruency.
*Psychonomic Bulletin & Review*, In press, 1–12.Google Scholar - Coppersmith, D., & Kumar, R. (2004). An improved data stream algorithm for frequency moments. In
*Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms*(pp. 151–156): Society for Industrial and Applied Mathematics.Google Scholar - Dale, R., Hindy, N. C., & Spivey, M. J. (2006). Feature-semantic gradients in lexical categorization revealed by graded manual responses. In
*Proceedings of the 28th Annual Meeting of the Cognitive Science Society: Lawrence Erlbaum*.Google Scholar - Dale, R., Kehoe, C., & Spivey, M. J. (2007). Graded motor responses in the time course of categorizing atypical exemplars.
*Memory & Cognition*,*35*(1), 15–28.CrossRefGoogle Scholar - Demšar, U., & Virrantaus, K. (2010). Space-time density of trajectories: Exploring spatiotemporal patterns in movement data.
*International Journal of Geographical Information Science*,*24*(10), 1527–1542.CrossRefGoogle Scholar - Di Crescenzo, A., & Longobardi, M. (2009). On cumulative entropies.
*Journal of Statistical Planning and Inference*,*139*(12), 4072–4087.CrossRefGoogle Scholar - Dillen, H., Phillips, J. G., & Meehan, J. W. (2005). Kinematic analysis of cursor trajectories controlled with a touchpad.
*International Journal of Human-Computer Interaction*,*19*(2), 223–239.CrossRefGoogle Scholar - Duran, N. D., Dale, R., & McNamara, D. S. (2010). The action dynamics of overcoming the truth.
*Psychonomic Bulletin & Review*,*17*(4), 486–491.CrossRefGoogle Scholar - Eilers, P. H., & Goeman, J. J. (2004). Enhancing scatterplots with smoothed densities.
*Bioinformatics*,*20*(5), 623–628.CrossRefPubMedGoogle Scholar - Engelbrecht, S. E. (2001). Minimum principles in motor control.
*Journal of Mathematical Psychology*,*45*(3), 497–542.CrossRefPubMedGoogle Scholar - Faulkenberry, T. J. (2014). Hand movements reflect competitive processing in numerical cognition.
*Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimentale*,*68*(3), 147.CrossRefPubMedGoogle Scholar - Fishbach, A., Roy, S. A., Bastianen, C., Miller, L. E., & Houk, J. C. (2007). Deciding when and how to correct a movement: discrete submovements as a decision-making process.
*Experimental Brain Research*,*177*(1), 45–63.CrossRefPubMedGoogle Scholar - Fisher, N.I. (1995).
*Statistical analysis of circular data*: Cambridge University Press.Google Scholar - Flash, T., & Henis, E. (1991). Arm trajectory modifications during reaching towards visual targets.
*Journal of Cognitive Neuroscience*,*3*(3), 220–230.CrossRefPubMedGoogle Scholar - Flodgren, G., Heiden, M., Lyskov, E., & Crenshaw, A. G. (2007). Characterization of a laboratory model of computer mouse use—applications for studying risk factors for musculoskeletal disorders.
*Applied Ergonomics*,*38*(2), 213–218.CrossRefPubMedGoogle Scholar - Freeman, J. B. (2016). Mouse tracker publications. http://www.mousetracker.org/publications/.
- Freeman, J. B., & Ambady, N. (2010). Mousetracker: Software for studying real-time mental processing using a computer mouse-tracking method.
*Behavior Research Methods*,*42*(1), 226–241.CrossRefPubMedGoogle Scholar - Freeman, J., Dale, R., & Farmer, T. (2011). Hand in motion reveals mind in motion.
*Frontiers in Psychology*,*2*, 59.CrossRefPubMedPubMedCentralGoogle Scholar - Friedman, J., Brown, S., & Finkbeiner, M. (2013). Linking cognitive and reaching trajectories via intermittent movement control.
*Journal of Mathematical Psychology*,*57*(3), 140– 151.CrossRefGoogle Scholar - Georgopoulos, A. P., Kalaska, J. F., & Massey, J. T. (1981). Spatial trajectories and reaction times of aimed movements: Effects of practice, uncertainty, and change in target location.
*Journal of Neurophysiology*,*46*(4), 725–743.PubMedGoogle Scholar - Gill, P. E., Murray, W., & Wright, M. H. (1981). Practical optimization.Google Scholar
- Glaholt, M. G., & Reingold, E. M. (2011). Eye movement monitoring as a process tracing methodology in decision-making research.
*Journal of Neuroscience, Psychology, and Economics*,*4*(2), 125.CrossRefGoogle Scholar - Good, I. J. (1963). Maximum entropy for hypothesis formulation, especially for multidimensional contingency tables.
*The Annals of Mathematical Statistics*, 911–934.Google Scholar - Gowayyed, M. A., Torki, M., Hussein, M. E., & El-Saban, M. (2013). Histogram of oriented displacements (HOD): Describing trajectories of human joints for action recognition. In
*Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence*(pp. 1351–1357): AAAI Press.Google Scholar - Hauk, O., Davis, M. H., Ford, M., Pulvermüller, F., & Marslen-Wilson, W. D. (2006). The time course of visual word recognition as revealed by linear regression analysis of ERP data.
*Neuroimage*,*30*(4), 1383–1400.CrossRefPubMedGoogle Scholar - Hawkins, G., Brown, S. D., Steyvers, M., & Wagenmakers, E.-J. (2012). Decision speed induces context effects in choice.
*Experimental Psychology*.Google Scholar - Hehman, E., Stolier, R. M., & Freeman, J. B. (2015). Advanced mouse-tracking analytic techniques for enhancing psychological science.
*Group Processes & Intergroup Relations*,*18*(3), 384– 401.CrossRefGoogle Scholar - Hogan, N., & Sternad, D. (2007). On rhythmic and discrete movements: Reflections, definitions and implications for motor control.
*Experimental Brain Research*,*181*(1), 13–30.CrossRefPubMedGoogle Scholar - Hwang, F., Keates, S., Langdon, P., & Clarkson, J. (2005). A submovement analysis of cursor trajectories.
*Behaviour & Information Technology*,*24*(3), 205–217.CrossRefGoogle Scholar - Johnson, A., Mulder, B., Sijbinga, A., & Hulsebos, L. (2012). Action as a window to perception: Measuring attention with mouse movements.
*Applied Cognitive Psychology*,*26*(5), 802– 809.CrossRefGoogle Scholar - Jonsen, I. D., Flemming, J. M., & Myers, R. A. (2005). Robust state-space modeling of animal movement data.
*Ecology*,*86*(11), 2874–2880.CrossRefGoogle Scholar - Kapsouras, I., & Nikolaidis, N. (2014). Action recognition on motion capture data using a dynemes and forward differences representation.
*Journal of Visual Communication and Image Representation*,*25*(6), 1432–1445.CrossRefGoogle Scholar - Kapur, J. N., & Kesavan, H. K. (1992).
*Entropy optimization principles and their applications*: Springer.Google Scholar - Koop, G. J. (2013). An assessment of the temporal dynamics of moral decisions.
*Judgment and Decision Making*,*8*(5), 527–539.Google Scholar - Lazebnik, S., Schmid, C., Ponce, J., & et al. (2009). Spatial pyramid matching.
*Object Categorization: Computer and Human Vision Perspectives*,*3*(4).Google Scholar - Lucero, J. C., & Koenig, L. L. (2000). Time normalization of voice signals using functional data analysis.
*The Journal of the Acoustical Society of America*,*108*(4), 1408–1420.CrossRefPubMedGoogle Scholar - McClintock, B. T., King, R., Thomas, L., Matthiopoulos, J., McConnell, B. J., & Morales, J. M. (2012). A general discrete-time modeling framework for animal movement using multistate random walks.
*Ecological Monographs*,*82*(3), 335– 349.CrossRefGoogle Scholar - McRae, K., De Sa, V. R., & Seidenberg, M. S. (1997). On the nature and scope of featural representations of word meaning.
*Journal of Experimental Psychology: General*,*126*(2), 99.CrossRefGoogle Scholar - Meyer, D. E., Abrams, R. A., Kornblum, S., Wright, C. E., & Keith Smith, J. (1988). Optimality in human motor performance: Ideal control of rapid aimed movements.
*Psychological Review*,*95*(3), 340.CrossRefPubMedGoogle Scholar - Morett, L. M., & Macwhinney, B. (2013). Syntactic transfer in English-speaking Spanish learners.
*Bilingualism: Language and Cognition*,*16*(01), 132–151.CrossRefGoogle Scholar - Mottet, D., & Bootsma, R. J. (1999). The dynamics of goal-directed rhythmical aiming.
*Biological Cybernetics*,*80*(4), 235–245.CrossRefPubMedGoogle Scholar - Norris, D., & Kinoshita, S. (2008). Perception as evidence accumulation and Bayesian inference: Insights from masked priming.
*Journal of Experimental Psychology: General*,*137*(3), 434.CrossRefGoogle Scholar - O’Really, C., & Plamondon, R. (2011). Can computer mice be used as low-cost devices for the acquisition of planar human movement velocity signals?
*Behavior Research Methods*,*43*(1), 229–238.CrossRefGoogle Scholar - Phillips, J. G., & Triggs, T. J. (2001). Characteristics of cursor trajectories controlled by the computer mouse.
*Ergonomics*,*44*(5), 527–536.CrossRefPubMedGoogle Scholar - Plamondon, R. (1995). A kinematic theory of rapid human movements.
*Biological Cybernetics*,*72*(4), 295–307.CrossRefPubMedGoogle Scholar - Plamondon, R., & Alimi, A. M. (1997). Speed/accuracy trade-offs in target-directed movements.
*Behavioral and Brain Sciences*,*20*(02), 279–303.PubMedGoogle Scholar - Ramsay, J.O. (2006).
*Functional data analysis*: Wiley Online Library.Google Scholar - Rao, M., Chen, Y., Vemuri, B. C., & Wang, F. (2004). Cumulative residual entropy: A new measure of information.
*IEEE Transactions on Information Theory*,*50*(6), 1220–1228.CrossRefGoogle Scholar - Resulaj, A., Kiani, R., Wolpert, D. M., & Shadlen, M. N. (2009). Changes of mind in decision-making.
*Nature*,*461*(7261), 263–266.CrossRefPubMedPubMedCentralGoogle Scholar - Rips, L. J., Shoben, E. J., & Smith, E. E. (1973). Semantic distance and the verification of semantic relations.
*Journal of Verbal Learning and Verbal Behavior*,*12*(1), 1–20.CrossRefGoogle Scholar - Shadmehr, R. (2005).
*The computational neurobiology of reaching and pointing: a foundation for motor learning*: MIT Press.Google Scholar - Shadmehr, R., & Mussa-Ivaldi, S. (2012).
*Biological learning and control: how the brain builds representations, predicts events, and makes decisions*: MIT Press.Google Scholar - Shimazaki, H., & Shinomoto, S. (2007). A method for selecting the bin size of a time histogram.
*Neural Computation*,*19*(6), 1503–1527.CrossRefPubMedGoogle Scholar - Smith, M. W., Sharit, J., & Czaja, S. J. (1999). Aging, motor control, and the performance of computer mouse tasks.
*Human Factors: The Journal of the Human Factors and Ergonomics Society*,*41*(3), 389–396.CrossRefGoogle Scholar - Song, J.-H., & Nakayama, K. (2009). Hidden cognitive states revealed in choice reaching tasks.
*Trends in Cognitive Sciences*,*13*(8), 360–366.CrossRefPubMedGoogle Scholar - Spivey, M. J., & Dale, R. (2006). Continuous dynamics in real-time cognition.
*Current Directions in Psychological Science*,*15*(5), 207–211.CrossRefGoogle Scholar - Tang, R., & Müller, H.-G. (2008).
*Pairwise curve synchronization for functional data*: Biometrika.Google Scholar - Van der Wel, R., Eder, J., Mitchel, A., Walsh, M., & Rosenbaum, D. (2009). Trajectories emerging from discrete versus continuous processing models in phonological competitor tasks: A commentary on Spivey, Grosjean, and Knoblich (2005).
*Journal of Experimental Psychology. Human Perception and Performance*,*35*(2), 588.Google Scholar - Voss, A., & Voss, J. (2008). A fast numerical algorithm for the estimation of diffusion model parameters.
*Journal of Mathematical Psychology*,*52*(1), 1–9.CrossRefGoogle Scholar - Walker, N., Meyer, D. E., & Smelcer, J. B. (1993). Spatial and temporal characteristics of rapid cursor-positioning movements with electromechanical mice in human-computer interaction.
*Human Factors: The Journal of the Human Factors and Ergonomics Society*,*35*(3), 431–458.CrossRefGoogle Scholar - Wang, F., & Vemuri, B. C. (2007). Non-rigid multi-modal image registration using crosscumulative residual entropy.
*International Journal of Computer Vision*,*74*(2), 201–215.CrossRefPubMedPubMedCentralGoogle Scholar - Wang, J., Sun, X., She, M. F., Kouzani, A., & Nahavandi, S. (2013). Unsupervised mining of long time series based on latent topic model.
*Neurocomputing*,*103*, 93–103.CrossRefGoogle Scholar - White, C. N., Ratcliff, R., Vasey, M. W., & McKoon, G. (2010). Using diffusion models to understand clinical disorders.
*Journal of Mathematical Psychology*,*54*(1), 39–52.CrossRefPubMedPubMedCentralGoogle Scholar - Yap, M. J., Balota, D. A., Tse, C. -S., & Besner, D. (2008). On the additive effects of stimulus quality and word frequency in lexical decision: evidence for opposing interactive influences revealed by RT distributional analyses.
*Journal of Experimental Psychology: Learning, Memory, and Cognition*,*34*(3), 495.PubMedGoogle Scholar