Tatool: A Java-based open-source programming framework for psychological studies
Tatool (Training and Testing Tool) was developed to assist researchers with programming training software, experiments, and questionnaires. Tatool is Java-based, and thus is a platform-independent and object-oriented framework. The architecture was designed to meet the requirements of experimental designs and provides a large number of predefined functions that are useful in psychological studies. Tatool comprises features crucial for training studies (e.g., configurable training schedules, adaptive training algorithms, and individual training statistics) and allows for running studies online via Java Web Start. The accompanying “Tatool Online” platform provides the possibility to manage studies and participants’ data easily with a Web-based interface. Tatool is published open source under the GNU Lesser General Public License, and is available at www.tatool.ch.
KeywordsCognitive training Experimental software Java Open-source Online experiments
Various software packages for conducting behavioral experiments exist—for example, E-Prime (Schneider, Eschman, & Zuccolotto, 2002), Inquisit (Draine, 1989), and SuperLab (Haxby, Parasuraman, Lalonde, & Abboud, 1993). However, the majority of this experimental software is proprietary, and thus is often expensive and not extensible to the demands encountered in the wide range of computer-based psychological research. There are also a growing number of freely available or open-source experimental software packages (e.g., Psychophysics Toolbox for MATLAB, Brainard, 1997; Pelli, 1997; PyEPL, Geller, Schleifer, Sederberg, Jacobs, & Kahana, 2007; Functional Measurement Builder Suite, Mairesse, Hofmans, & Theuns, 2008; TScope, Stevens, Lammertyn, Verbruggen, & Vandierendonck, 2006; and PsyToolkit, Stoet, 2010), which are more cost-effective and often more flexible. However, most of these software packages generate platform-dependent code that is suitable solely for specific operating systems (e.g., Microsoft Windows or Mac OS). Moreover, some software packages can only be used for offline research in the lab. Tatool is an open-source programming framework based on Java, and hence is extensible and platform-independent. Tatool was developed specifically with a focus on facilitating cognitive training research, but it can be used for setting up experiments and questionnaires as well, both online and offline. With Tatool, one compiled version of the experimental tasks can be distributed online to all participants. This is particularly useful for cognitive training studies, in which it is often required that training sessions run self-administered from participants’ home computers with different operating systems. Moreover, Tatool provides multiple features that are crucial for conducting training studies (e.g., session schedulers, adaptive training algorithms, and a Web-based interface to monitor participants’ commitment), which will be described in more detail below.
Tatool is published open source under the GNU Lesser General Public License. By editing the source code of Tatool, the software can be extended to virtually every need that may be encountered during the technical development of psychological tasks or studies. Because Tatool is open source, this also implies that fellow researchers can—and are invited to—contribute their own ideas to the Tatool main release in order to optimize the framework for different fields of research. The code of training or experimental tasks can be published under the GNU General Public License, so researchers can easily exchange tasks or code snippets without compromising their authorship of a specific task.
Tatool is based on the programming language Java, which is an object-oriented language with a common syntax. Furthermore, a Java-based programming framework entails three key benefits. First, Java is a portable language. Therefore, Tatool applications can be written on one operating system (e.g., Windows) and run on every other operating system, such as Mac OSX or Linux, without the need to change the code. Hence, this tool is truly platform independent. Second, the Java Runtime Environment (JRE) necessary for executing Java applications can be downloaded for free and is easily installed. Tatool requires Java version 1.6 or later. To date, many applications in digital everyday life use Java technology, which means that most participants will probably have the JRE already installed. Third, the Java Web Start technology included in the JRE allows for the online distribution of the Tatool application client. With a few mouse clicks, the client is installed on a local computer. After installation, the application runs on the local computer and does not require an Internet connection anymore, avoiding typical issues of Internet testing (e.g., random noise produced by the Internet connection). Hence, it allows for running experiments or training studies online without sacrificing the advantages of an offline application (see also Schmiedek, Bauer, Lövden, Brose, & Lindenberger, 2010). Java Web Start can also be used to facilitate the distribution of software updates. When starting the local application, Java Web Start can optionally search for changes to the online code and update the local version accordingly.
The Tatool framework can be used with different levels of complexity, depending on the user’s programming skills. For experimenters or software developers with at least moderate prior knowledge of Java or object-oriented programming, the Tatool framework provides an extensive application programming interface (API). It comprises a large number of Java classes (in simplified terms, bundles of functions) and methods that are useful in the context of psychological studies in general, and in particular for computer-based training studies. The full API documentation, including a comprehensive list of Tatool’s classes and methods, is accessible online via www.tatool.ch/javadoc. Once programmed in Java, study components (e.g., instructions, questionnaires, and tasks) are arranged in module files written in XML (“Extensible Markup Language”), which is a text format that is readable by both humans and machines. XML documents follow a tree structure, consisting of a single root element that branches into one or more child elements. The grammar of the XML document is constrained by a specific set of rules, called the schema language. Tatool utilizes the XML schema provided by the open-source Spring framework (www.springsource.org). Due to its easily comprehensible format, XML can be viewed as a translator between the experimenter and Tatool. In the module file, the parameters of an experiment, such as the display duration of stimuli or the number of trials, can be edited without having to change the Java code (see the How to Get Started section below). Therefore, it offers the opportunity for the properties of already-implemented tasks to be customized by researchers or students without or with only few programming skills. Module files can be opened and started with the Tatool application client from online and from local sources.
Data produced during the execution of a module are stored in a local database coming with Tatool, from which they can be exported as comma-separated value (CSV) files. These files are composed of one row per trial and several columns for the variables of interest, which can be virtually any stimulus properties or (measurable) variables. The CSV file can be opened with a spreadsheet program such as Microsoft Excel, in which data can be aggregated with a few mouse clicks via pivot tables. The CSV file can also be viewed with any text processor or imported into statistic programs such as SPSS. To facilitate data aggregation, the column headers in the CSV file are named according to definitions made in the XML module file and are, therefore, meaningful to the experimenter.
Data can be exported either locally, on the client computer, or online, to an FTP server or a Web server running Tatool Online, which allows for managing studies via a Web interface. This component can be installed on any Web server that supports PHP and MySQL. An example installation can be viewed at www.tatool.ch. Experimenters can create studies, add groups, and assign participants to these groups. Different experimenter profile settings would facilitate complex study designs, such as double-blinding (i.e., when one or more experimenters must not be informed about the identity and group membership of participants). Furthermore, Tatool Online provides helpful features for studies that require participants to do the experimental tasks at home, as is the case in a growing number of training studies. The data can be uploaded automatically, reducing data loss due to possible technical problems. Importantly, data are also stored locally in an embedded database (HSQLDB). Hence, in the case of technical problems (e.g., a loss of Internet connection), data can be uploaded later or exported locally. Tatool Online processes uploaded data in real time and lists all participants and their sessions, including the date and time of the last upload, session duration, and custom information (e.g., performance and reaction time). This enables experimenters to gain a quick overview of the participants’ commitment and to intervene if necessary. Moreover, Tatool Online allows for monitoring the overall progress of a study by providing basic descriptive statistics and simple graphs on the aggregated level, such as performance means and minimum and maximum values. The experimenter can either access the data via Tatool Online or download it from the FTP server hosting the Tatool Online installation.
Tatool’s architecture matches the design of typical psychological experiments. The core concept of Tatool’s architecture is the module, which represents the experiment or study (e.g., a cognitive training regimen, an experimental design, or a series of questionnaires). A module comprises multiple elements that can be of a number of types: list elements, compound elements, or executable elements. The list and compound elements serve to group executable elements, and can be nested within each other. List elements set the order in which elements listed in it are executed. Compound elements contain a primary and a secondary element and can be used as a basis for the implementation of dual tasks (e.g., a Brown–Peterson task [Brown, 1958] or a complex span task [Daneman & Carpenter, 1980]). First, the primary element is executed (e.g., the display of a list of words to memorize). During the following execution of the secondary element (e.g., a distraction task), the primary element is suspended until the secondary element is completed. Afterward, the primary element is executed again (e.g., the display of another list of words or of a prompt to recall the words memorized before). Only the third type of element, the executable element, can contain an executable. The executable is the implementation (i.e., the Java class) of the actual task that the participants have to do, such as the memorization and recall of a list of words, in the example above. Every executable element can only contain precisely one executable. Therefore, modules including more than one task will comprise several executable elements nested within list and/or compound elements.
The module hierarchy is defined in an XML file. It can be opened with the Tatool application, which then saves the file’s contents in a local Java database (HyperSQL) used within Tatool. With every execution of a module, Tatool creates a session and saves the data produced during this session in the same database. Each execution of an executable can be recorded and stored as a trial within the given session. Figure 3b shows the different elements of a module as they are reflected in the data view (i.e., data output).
So far, Tatool has been used for programming a wide range of tasks in the form of executables—for example, dual tasks (complex span or Brown–Peterson), visual matching tasks (e.g., face matching), task switching, visual search, a trivia quiz, and several questionnaires. Moreover, we implemented several handlers for different purposes, two examples of which we will present here.
Adaptive training algorithm
The adaptive training algorithm is implemented as a handler, which acts across all phases during module execution and can be used by any executable. At the beginning of a session, it loads the level achieved in the last session. Before the task is executed (i.e., in the “pre_process” phase), it displays the current level of difficulty in the status panel (in the top of Fig. 2). After execution of the executable, the algorithm calculates the participant’s performance and, depending on the number of trials completed, checks whether the level of task difficulty has to be increased or decreased. At the end of the session, the algorithm saves the level of difficulty, thereby allowing participants to start the next session on the same level.
EEG trigger handler
In another experiment, we recorded task-related brain activity with electroencephalography (EEG). To allow communication between the Tatool application that ran the cognitive test battery and the EEG recording device, we needed a parallel port interface. Therefore, we used the Java native interface, which utilizes C code to interact with the parallel port and any hardware attached to it. In our case, Tatool sent signals to the pins of the parallel port that were, in turn, read by the EEG device that set the markers needed for the analysis of the EEG recordings. To be able to set markers at stimulus onset and when the participants responded to this stimulus, the handler acted during the execution of the executable (i.e., in the “execute_executable” phase).
How to get started
Tatool can be used with different levels of complexity that come with different levels of flexibility. At the top level of complexity and flexibility, users can extend Tatool to meet virtually every possible requirement and can contribute these extensions to the Tatool main release. This requires a deep knowledge of Java, the use of subversion, and XML. Most prospective users will probably use Tatool to program specific experimental or training tasks and to set up studies, which requires basic knowledge of Java and XML. In this case, the level of complexity is moderate, but the level of flexibility is still very high, because the Tatool framework comprises predefined functions that meet many of the needs of computer-based experimental psychology. To program tasks with the Tatool framework, any Java integrated development environment (IDE) can be used—for example, the open-source Eclipse IDE (www.eclipse.org). In the Tatool documentation available at www.tatool.ch, we provide instructions on how to set up the development environment and step-by-step tutorials on how to program tasks within the Tatool framework and how to set up the XML module files to run a study. Tatool can be run with or without Tatool Online. Little knowledge of MySQL database queries is required to install Tatool Online on a Web server.
At its lowest level of complexity and flexibility, users can easily modify the parameters of already-programmed experiments or training implementations (e.g., number of trials, display durations of the stimuli, or performance thresholds for adaptive training algorithms) within the module file, written in XML. In the following example, we show how to modify single parameters in a working experimental task that is included in the Tatool demo client available at www.tatool.ch/demo.htm. In this Stroop-like task (Stroop, 1935), the words GREEN and BLUE are randomly presented on the screen in the color green or blue, and the correct color has to be identified as quickly and accurately as possible by a keypress. To modify the experimental parameters of this task, the module file for the demo client has to be downloaded from www.tatool.ch/download.htm. The text file, with a .xml extension, can be opened with any text editor. Next, the Tatool demo client (www.tatool.ch/demo.htm) has to be downloaded, following the instructions. Instead of (or in addition to) adding the module file from Tatool Online, the option “Module from local file” should be selected, in order to open the previously downloaded module file from its location on the hard drive.
Modifiable parameters of the example module file
Duration in milliseconds of a pause (i.e., a blank screen) between the study components
Defines the number of iterations of elements within a list (i.e., number of trials)
Number of trials that are used to calculate the user’s accuracy for the adaptive training algorithm
If the accuracy in a specified number of trials (sampleSize) is above this value, task difficulty will be increased.
Integer between 0 and 100 (inclusive)
If the accuracy in a specified number of trials (sampleSize) is below this value, task difficulty will be decreased.
Integer between 0 and 100 (inclusive)
Limitations and future directions
Tatool offers a multitude of features and high flexibility. Importantly, Tatool is a programming framework, and not a graphical experiment builder. Therefore, Tatool’s high flexibility entails that developing tasks with Tatool is code-heavy, and prospective users thus will need basic knowledge of Java and XML. Several well-written tutorials on Java are available in the form of books or on the Web for free. On our webpage, we also offer step-by-step tutorials for novices using the Tatool framework. An accompanying discussion board on our webpage allows users to exchange code snippets and experimental tasks and to get help from the community concerning programming issues. Once an application is programmed in Java, experimental parameters can easily be modified via module files written in XML. To further facilitate the modification of experimental parameters, in the near future we plan to implement a graphical user interface for setting up module files more quickly.
The Tatool programming framework was developed mainly to facilitate cognitive training research, but it can be used for any computer-based experiment. For specific paradigms, however, specialized software packages may be more suitable. For example, the Functional Measurement Experiment Builder Suite (Mairesse et al., 2008), for conducting experiments using the functional measurement paradigm, or WebExp (Keller, Subahshini, Mayo, & Corley, 2009), for conducting simple Web experiments, are both Java-based applications, and therefore are also platform independent, but they require a lot less prior programming knowledge.
Another possible limitation of Tatool concerns timing. Generally, the timing implementation in Java does not offer the same accuracy offered by programming languages that generate native machine code or external timing hardware. The accuracy of timing in Tatool relies heavily on the timer precision provided by the underlying operating system. The nanoTime method, available since Java 1.5, uses the highest resolution clock available on the platform, and while its return value is in nanoseconds, the update resolution is typically only in microseconds. On modern hardware and operating systems, the Java methods can deliver accuracy and precision in the microsecond range. Java’s timing performance can be improved by using the Java native interface, which calls functions provided by a low-level programming language such as C (cf. the implementation of the EEG trigger handler above).
To date, Tatool is available in German and in English, but translations into other languages might be required. Tatool is internationalized following the coding practice of the i18n approach. This means that strings (e.g., names of buttons) are stored externally in separate resource files that can be translated with minor effort.
Tatool is a Java-based open-source programming framework that comprises a large library of functions useful for running psychological experiments. It is particularly useful for cognitive training studies and boasts multiple features, such as configurable training schedules, adaptive training algorithms, and an optional Web-based interface that supports monitoring of participants’ commitment. It has high portability, can be flexibly adapted to virtually every requirement, and is published open source, and hence is available for free. Future plans include extending the documentation and tutorials already available and developing a graphical interface to facilitate the creation and modification of module files.
This work was supported by a grant to the first author from the Forschungskredit of the University of Zurich. We thank Nicole Cruz, Simone Eberhart, Julia Herkert, and the experimenters of past studies for testing earlier versions of Tatool and for their valuable feedback. We also thank Klaus Oberauer for his helpful comments on an earlier version of the manuscript.