To investigate how people perceive stimuli in terms of similarities, holistic approaches such as projective-mapping or sorting tasks can be used. For the projective-mapping task, subjects are asked to position stimuli on a plane according to their similarities (Risvik, McEwan, Colwill, Rogers, & Lyon, 1994). The closer together two stimuli are positioned, the more similar to each other they are perceived to be. In terms of data collected, each subject provides a dissimilarity matrix between the stimuli, based on a quantitative measure—namely, the distance between the stimuli. This technique is also known as the spatial arrangement procedure (Goldstone, 1994). For the sorting task, subjects are asked to gather stimuli into clusters according to their similarities. Two stimuli belong to a same group if they are perceived as similar. In terms of data collected, each subject provides a dissimilarity matrix between the stimuli based on a binary qualitative measure—namely, 0 if two stimuli belong to a same group, and 1 otherwise.

A common feature between these two methods is that subjects assess a set of stimuli on the basis of their perceived similarities, hence the concept of similarity-based methods (Valentin, Chollet, Lelièvre, & Abdi, 2012). These two methods can be considered holistic in the sense that each stimulus is considered as a whole and not as a sum of predefined components. These methods are widely used in different fields such as sensory analysis and marketing, in which stimuli are globally assessed through human senses.

The objectives of such methods are to provide a representation of the stimuli based on a group of subjects, and eventually to compare the individual assessments within that representation.

In terms of data collection, projective-mapping and sorting tasks can be easily performed with various types of stimuli, due to the different commercial and noncommercial software available. For instance, EyeQuestion (Logic8, 2015), Fizz version 2.5.0 (Biosystème, 2015), and TimeSens (ChemoSens, 2015) are designed to handle stimuli in form of simple texts and images; TCL-LabX (Gaillard, 2009) is a free software product designed to handle sound stimuli only, but that has to be programmed for texts or images; and NappingPlayer (Robitza, Helpiquet, Kitzinger & Hlavacs, 2013) is a free video player for Android tablets designed to conduct Napping® (Pagès, 2005) for videos only. With the current software solutions, researchers have to choose their softwares according to their types of stimuli.

Nevertheless, two main drawbacks remain with the existing software. First, they can only handle some but not all stimuli, and are restricted to using either texts, images, sounds, or videos. Consequently, this limits the application of these methods to various research fields. Second, only the final configurations provided by subjects are collected. Consequently, these solutions do not allow recording of the different steps followed by the subjects to create their final configuration; indeed, a given dissimilarity matrix provided by two subjects may reflect two different behaviors during the task (as we show later in this article). This sequence of steps, which can be assimilated to the cognitive process used by the subject to perform the task, can bring a new perspective to the collected data.

In this article, we describe a tool to collect the individual process that leads a subject to the final dissimilarity matrix, and we describe an R package to illustrate these individual processes. This tool, named Holos, is the combination of a software product for collecting data and a server for storing data. Compared to most commercial software, Holos can handle several kinds of stimuli: texts/concepts (i.e., words or statements), images, sounds, and videos. The software product is an Android application with a tactile human machine interface in which subjects can conduct experiments using a tablet. During the experiment, as the subject moves an icon by dragging it across the tablet screen, the server records the trajectory of the subject’s finger (i.e., the trajectory of the stimulus) in the form of digit-tracking data, akin to eyetracking data. The digit-tracking data can be analyzed with a function available in the SensoMineR package (Lê & Husson, 2008). Interestingly, the server is a free collaborative environment, in which any researchers with a Holos account can partially or totally share their study resources within the scientific community. In the next sections, we will present in detail Holos and its different features; we will end the article by presenting an exemplary experiment.

Holos

Holos can be defined as a software solution that allows researchers to measure an individual’s real-time process during an experiment based on holistic similarity assessments between stimuli. Holos was designed to handle four types of stimuli: concepts, images, sounds, and videos.

Overall structure

Figure 1 represents the working process of the Holos environment, which can be broken down into three components. First, a server provides a space in which researchers can design their experiments. On this server, researchers can also share their experiment resources (stimuli, protocol, etc.). In that sense, Holos can be defined as a collaborative environment dedicated to holistic data, from methods based on measurements of between-stimulus similarities or differences. The second component is an Android application combined with a tactile interface, with which the subjects can perform the experiment. Holos records the streaming x,y coordinates of the stimuli while they are dragged by the subject across the tactile tablet during the task. This feature, which is the most important characteristic of Holos, generates data that allow researchers to review their experiments and then to better understand the behavior of their subjects during the task. These data have been called digit-tracking data, since Holos was developed for tactile tablets; in other words, stimuli are tracked as the subject’s digit drags them while performing the task. The third component is a function available in the SensoMineR package to analyze such data. When the holistic task consists of collecting quantitative similarity/dissimilarity data (i.e., when the task is a projective-mapping task or one of its variants), the digit-tracking data can be statistically analyzed to obtain a graphical representation that could be assimilated to the cognitive process of a subject during the task.

Fig. 1
figure 1

Representing steps in the working process of the Holos environment

Full size image

Installation

Each component of Holos can be considered independently. First, researchers can access the server at the platform http://napping.agrocampus-ouest.fr by creating an account. Second, subjects can download the Android application on their tablet device. This application, named NSubject, is also available at the website http://napping.agrocampus-ouest.fr. Its installation is similar to that of other applications in the Google Play store. Third, to analyze the digit-tracking data, researchers can use the SensoMineR package (Lê & Husson, 2008), a free R package dedicated to the analysis of sensory data.

Designing an experiment

Experiments are designed by the researchers via their account on the website http://napping.agrocampus-ouest.fr. Practically, in terms of the experiment organization, researchers can manage their stimuli and protocols, and the recruitment of their subjects for their experiment. A user manual is available on the website.

Specifying the stimuli

As we mentioned previously, Holos can handle four formats of stimuli (viz., text, image, audio and video) as Holos is dedicated to the study of concepts, images, sounds and videos. For a given experiment, stimuli must be the same type/format. When stimuli are images, as the stimuli will be represented as small icons during the experiment on the tablet device, the researcher has to import the stimuli and their thumbnails as well. Once stimuli are uploaded, the researcher can change the names that will appear to the subject during the experiment, directly below the thumbnails. By default, these are the names of the stimuli that have been imported. They can be replaced, for instance, by random three-digit numbers or by letters.

Specifying the protocol

The protocol corresponds to the instructions that will pop up on a splash screen on a tablet at the beginning of the experiment. Because Holos can handle several kinds of tasks (projective mappings, sorting tasks, etc.), the protocol should be specified carefully.

Specifying the subjects

Because Holos is a Web application, the researcher has the possibility to send an e-mail to a list of subjects to invite them to join a given experiment, and to perform the experiment whenever they want, on their own tablet. To do so, the researcher has to enter the list of subjects to be contacted. Once this list is filled out, an e-mail is sent to the subjects to inform them that they are invited to an experiment. The invitation is associated with an ID number for the experiment, which is mandatory for the subject to perform the tasks.

Performing an experiment as a subject

To perform an experiment, the subject has to open the NSubject Android application. The subject has to provide their name, e-mail address, and the ID in the experiment. Once the application is opened, the protocol of the experiment appears in a splash screen, and the stimuli are represented in the form of icons. These icons are randomly displayed on the left side of the screen. This arrangement is different from one subject to the next. Each icon can be dragged by using one’s finger. A double-click on an icon induces an action that depends on the type of stimulus; it makes the image appear in the case of image stimuli, plays a sound or video in the case of audio and video stimuli, or opens a screen on which the definition of the concept appears in the case of text stimuli. When the task is over, subjects can write down information to describe the stimuli.

Digit-tracking data and their analysis

During the task, as the subject moves the stimuli on the tablet, the coordinates are recorded. More precisely, for a given subject, the application generates a matrix of dimensions I × 2 each time a stimulus has been moved, in which I is the number of stimuli and 2 the number of coordinates recorded (i.e., the x-axis and y-axis). By recording all these coordinates, it is possible to reconstitute the trajectories of all the stimuli. These data are automatically stored in the Holos server.

Once the experiment is over (i.e., once all subjects have performed the task), the researcher can access the digit-tracking data on the server. To analyze them, the folder obtained during the experiment must be downloaded on the researcher’s computer. Digit-tracking data can be analyzed using the SensoMineR package (version available at the website http://sensominer.free.fr/). The holos function of this package allows researchers to analyze such data. Since the data depend on the holistic method used to collect them, the main argument of this function is the type of task that was performed. When the holistic task is data collection based on quantitative similarities/dissimilarities (i.e., when the task is projective mapping or one of its variants), representations of the cognitive processes can be obtained. This is not the case when the task is a sorting task or one of its variants. The analysis of data obtained with a projective-mapping task is described below.

The holos function analyzes the final configurations. In the matrix containing the final configurations, each line corresponds to a stimulus, and the columns correspond to the final configurations provided by the subjects (i.e., the x,y coordinates provided by each subject at the end of the experiment; see Table 1). Such a matrix is analyzed with a multiple-factor analysis in which each group is composed of the two columns relative to a subject. This analysis provides two major pieces of information: the representation of the stimuli over all subjects, and second, the representation of each subject.

Table 1 Dataset containing the final configurations
Full size table

The holos function analyzes the digit-tracking data. In the matrix containing the digit tracking, each line corresponds to a stimulus, and the columns correspond to the x,y coordinates of the stimuli during the steps as the subjects perform the experiment (see Table 2). Such a matrix is analyzed with a multiple-factor analysis in which each group is composed of the two columns relative to one step, followed by a subject. In this analysis, only the final configurations are defined as the active groups, whereas all of the intermediate steps are defined as supplementary groups. This analysis provides, for each subject, a representation that allows an experimenter to understand how this subject has perceived the stimuli throughout the experiment. This representation can be assimilated to the mental processing used by this subject to perform the task up to the final configuration. The holos function of SensoMineR provides other pieces of information to summarize the digit-tracking data for each subject (e.g., the number of steps, duration of the task, and duration of the learning phase).

Table 2 Dataset containing the stimuli trajectories
Full size table

Experiment

This experiment was realized with stimuli corresponding to cards created with a factorial design. The aims of this experiment were (1) to understand which factors were first perceived by subjects, and (2) to understand how the subjects perceived the stimuli during the whole experiment (i.e., to determine the mental processing the subjects used during the task to provide their final configuration).

Method

Stimuli

A total of 16 cards were created with a factorial design (Cadoret et al. 2011). The following were the factors used: Background Color (orange vs. yellow), Shape Color (blue vs. green), Contour (dotted vs. continuous line), Shape (circle vs. square), Size (big vs. small), Position (top vs. bottom), and Background Pattern (shaded vs. plain). Figure 2 presents two of the 16 stimuli.

Fig. 2
figure 2

Illustration of two stimuli, C and I, created with a factorial design

Full size image

Protocol

Subjects performed a Napping task, a particular case of projective mapping. The protocol was the following one: “During this experiment, you are asked to position the stimuli on the screen in such a way that two stimuli are close if you perceive them as similar, and two stimuli are distant if you perceive them as different.”

Subjects

We used the data from five subjects.

Results

Final configurations

The five subjects produced different final configurations (see Fig. 3). Subject S1 separated the stimuli on the basis of their background color; subjects S2 and S5 used the shape color; subject S3 used both background and shape color; and only subject S4 used the contour to separate the stimuli.

Fig. 3
figure 3

Final configurations provided by the five subjects

Full size image

As is shown in Fig. 4a, the first dimension of variability among the cards is the shape color, and the second dimension is the background color. The first dimension, denoted Dim 1, opposes the cards with a green shape, on the left side, to those with a blue shape, on the right side. It explains 43.23 % of total variability of the data. The second dimension, denoted Dim 2, opposes the cards with orange backgrounds, on top, to the cards with yellow backgrounds, at the bottom. It explains 36.19 % of the total variability of the data.

Fig. 4
figure 4

a Representation of the stimuli. b Representation of the subjects (one subject is represented through its group of variables)

Full size image

The representation of the subjects involved in the experiment in Fig. 4b has to be interpreted jointly with the previous representation of the stimuli. For instance, S1 gets a high value on Dim 2, meaning that the main variability on their final configuration is consistent with the background color. In other words, S1 separated the stimuli on the basis of the background color. S2 and S5 get a high value on Dim 1, since they separated the stimuli on the basis of the shape color. S3 gets a high value on both Dim 2 and Dim 1, since this subject separated the stimuli according to both background and shape color. Finally, S4 gets low values on both Dim 1 and Dim 2, since he separated the stimuli on the basis of the contour.

Cognitive processes

The representations of the cognitive processes of the subjects involved in the card experiment are shown in Fig. 5 (Lê et al. 2016). The representation of the cognitive process used by S1 shows that this subject performed 16 steps to obtain the final configuration. At the beginning of the experiment, the configurations of the cards (e.g., Step 1 or Step 2) get low values on Dim 2. Gradually, at the end of the experiment, the configurations of the cards (e.g., Step 15 or Step 16) get high values on Dim 2. In other words, the cognitive process used by this subject shows gradual perception of the background color in successive steps from the beginning to the end of the experiment.

Fig. 5
figure 5

Representations of the subjects’ cognitive processes

Full size image

The representation of the cognitive process used by S3 shows that this subject also took 16 steps to reach the final configuration. This subject perceived both the shape color and the background color throughout the experiment as separating the stimuli.

The representation of the cognitive process used by S4 shows that this subject conducted 16 steps to obtain the final configuration, but perceived neither the background color nor the shape color during the whole experiment.

The representations of the cognitive processes used by S2 and S5 are interesting. As we mentioned previously, these two subjects are very close in the representation of the subjects (see Fig. 4b), meaning that the ways that they positioned the stimuli are similar. Even if they provided the same final configuration, S2 and S5 did not use the same cognitive process when performing the task. First, S2 and S5 did not use the same number of steps to reach their final configuration: It took 16 steps for S2, and 26 steps for S5. Second, S2 and S5 did not use the same mental processing when they performed the task. Indeed, S2 gradually perceived the shape color in successive steps from the beginning to the end of the experiment, whereas S5 first perceived the background color (from Step 1 to Step 16), and then perceived the shape color (from Step 17 to Step 26).

Discussion on the experiment

This example highlights the importance of collecting digit-tracking data during an experiment based on similarity assessments between stimuli. Indeed, the sole analysis of the final configurations would have been misleading. Without the representation of the cognitive processes, we would have concluded that subjects S2 and S5 performed the task similarly and saw the same similarities and dissimilarities within the stimuli. This conclusion would have been incorrect, because subjects S2 and S5 behaved differently during their task: They did not use the same number of steps to reach their final configuration, nor did they use the same mental processing.

Conclusion

In the present article, we have presented Holos, an environment for holistic experiments based on similarity assessments between stimuli. We exposed the general features of this environment, and we described how the data it provides can reveal real-time individual processes during the task. By recording the x,y coordinates of the stimuli, Holos is the first software based on similarity measures that integrates subjects’ behavior into the data analysis step. The importance of recording these stimulus trajectories, called digit-tracking data, was highlighted with an exemplar experiment: Such data can notably reveal how two subjects can provide identical final configurations through two different cognitive processes.

Beyond the fact that Holos is a software product for collecting real-time individual process, Holos is also collaborative and based in a free environment. On the platform, researchers can access open study resources (stimuli, protocols, etc.) and benefit from this feature of Holos to conduct cross-cultural studies.

The environment still needs to be improved, in particular when researchers want to share their study resources. In terms of the stimuli, when an experiment involves videos, subjects may encounter problems, especially when the size of the video is large and a subject wants to play it back. In terms of methodologies, we want to add new ways of collecting data, and in particular the Q-sort, which allows researchers to study subjects’ subjectivities when they are confronted with a pool of stimuli.

Since the beginning of its development, dozens of experiments have been performed successfully within the Holos environment. Our hope is that Holos will be used by researchers from various fields, such as experimental psychology, cognitive science, marketing, and so forth.