1 Introduction

Over the last 15 years, smartphones have become highly relevant in society and social research. In addition to studies on smartphone use itself, smartphones have been used as data collection tools due to their ubiquity and pervasiveness in everyday life. However, researchers in communication science and related disciplines, such as (media) psychology or political science, are often interested in variables requiring different valid measurement methods. For example, attitudes towards certain issues can be measured in surveys, as these are relatively stable and reliable concepts that show little intra-individual variation and are easy to retrieve for respondents (cf. Bhattacherjee 2012, pp. 73–74). Smartphone use (mostly measured through frequency and duration) is usually also assessed retrospectively via self-reports in (online) surveys. Such data are, however, typically influenced by poor retrospective estimation and social desirability bias (cf. Schwarz and Oyserman 2001, p. 129). For this reason, the data differ quite significantly from those produced by less subjective, technologically assisted methods, such as logging, which are preferable in this case (cf. Parry et al. 2021, p. 1543). Other subjective variables, such as well-being or reactions to news articles, are also affected by such problems but cannot be assessed through logging. These need to be measured with self-reports but need to be completed as soon as possible after the situation of interest occurs. This can be accomplished with in-situ methods, such as the Experience Sampling Method (ESM, cf. Larson and Csikszentmihalyi 2014). All three methods of data collection have been implemented on smartphones before (e.g., cf. Johannes et al. 2020, pp. 8–9; Naab et al. 2018, p. 6), and smartphones have also been used for data donations of other multimodal information for quite some time (e.g., cf. Otto et al. 2022; Raento et al. 2009).

Many data collection apps in the past were programmed for individual studies and were not maintained or used afterward; only a few were published for other researchers to use. The published apps were either limited to particular measurement methods (e.g., cf. Menthal n.d.; Movisens 2022), required programming skills (cf. AWARE 2022), or were either commercial solutions or collected non-optional, sensitive data (e.g., GPS locations: cf. Murmuras n.d.). Furthermore, people’s willingness to participate in studies using passive observation methods like logging is already limited (cf. Struminskaya et al. 2021, pp. 448–451). Therefore, apps combining logging with other methods are bound to reduce willingness to participate even further. The GESIS-Leibniz Institute for the Social Sciences is currently working on an app dedicated to ESM data collection and another for assessing sensor data (logging, accelerometer, etc.), which tackle some of these problems (cf. GESIS n.d.). However, they do not combine self-reported and behavioral data assessments within a single app. As different methods are required for validly measuring smartphone use and other variables, it is necessary to establish a universal tool for research that implements all these methods. A new smartphone app is, therefore, ideally suited for this purpose.

This paper presents the Android and iOS app MART (Mobile Assessment Research Tool) to the communication and media research community. It qualifies as free and open-source software (FOSS), is customizable without programming knowledge, and combines all three measurement methods mentioned. With this app, smartphone use and other variables relevant to the social sciences can be assessed using the most appropriate method. Data collection occurs right on the device that study participants use daily, reducing the burden for them and the researchers.

In line with recommended open science practices (cf. Peter et al. 2020), the source code of MART was already made available via GitHub. Currently, the developers’ goal is to prepare a framework that enables any researcher in the field to use it. This paper describes MART’s use cases, functionality, backend mechanics and prospects.

2 Use cases

Many studies in communication science and adjacent fields combine multiple data collection methods (e.g., cf. Andrews et al. 2015; Bjørner 2016; Burt and Alhabash 2018; Naab et al. 2018). Given the participant burden of single methods like ESM alone (cf. Eisele et al. 2020, pp. 10–13), the combination of and transition between multiple methods in a study can be expected to increase this burden even further. MART reduces the participant burden as far as possible by allowing for combinations of multiple methods on a single device: the participant’s smartphone.

For example, MART enables researchers to first assess sociodemographic information and attitudes toward specific behaviors using a survey. Then, researchers can measure whether participants’ self-reported behaviors align with their attitudes using in-situ measurement through ESM. If a behavior is associated with smartphone use, the logging functionality adds an objective assessment of that behavior, which can be used to evaluate the validity of the ESM data.

Another possible application of MART is using only one or two of these measurement methods. While it is possible to conduct online surveys on platforms such as SurveyMonkey, ESM using dedicated apps (e.g., cf. Movisens 2022), and logging using custom solutions (e.g., cf. Toth and Trifonova 2021, pp. 3–4), MART can handle all these methods without switching between tools within and between studies. Consequently, it is an all-inclusive solution for combining methods and for single-method studies.

3 Functionality

MART was initially developed by nvii-media GmbH (2022) as part of a research project led by the author of this article. The project was funded by the German Research Foundation (DFG) and involved the development of the app and participant recruitment for a study that used it. This chapter describes the implementation of each of the three data collection methods enabled by MART. Other than differences in the implementation of logging/data donation, the Android and iOS versions of the app are identical.

3.1 Survey

The most basic functionality of the app is the survey, which largely corresponds to online surveys implemented on platforms like Unipark, Qualtrics, LimeSurvey, LamaPoll, or SoSci Survey. It is currently possible to set up pages containing text only (e.g., for briefings or question introductions) or four different types of items.

Radio buttons serve the purpose of selecting one element from a list of options. Alternatively, the same options can be displayed as a drop-down menu. Sliders can be used as an alternative to radio buttons, which is helpful if options are numeric. Checkboxes provide a list of options, but they enable multiple choice. Finally, text input can be used for the assessment of non-standardized information. All scales/answering options and default values can be customized. Items can be grouped in item batteries, within which their order can be randomized.

Additionally, there is a mechanism for time-limited questions (e.g., five seconds per item). This is used to assess habitual media use initiation using the Response-Frequency Measure of Media Habit (RFMMH) described by Naab and Schnauber (2016). These time-limited questions can provoke choices under time pressure to access unconscious tendencies (cf. Naab and Schnauber 2016, pp. 134–136).

3.2 ESM

All functions described in the previous section are also available for ESM questionnaires. Additionally, it is possible to set up a signaling schedule. In this case, a signal is equivalent to a notification on the smartphone screen encouraging the user to fill out an ESM questionnaire.

Time frame and intervals

First, researchers have to define the maximum number of ESM questionnaires a study participant should be able to complete. Then, they have to set a specific time frame during the day to enable signals (full hours and even number of hours, e.g., 8 am–10 pm). This time frame is then split into intervals of two hours each (e.g., 8 am–10 am, 10 am–12 am, 12 am–2 pm, etc.). Additional options will be added.

Schedules

In ESM research, different signaling schedules are used. Schedules are either signal-contingent, interval-contingent, or event-contingent (cf. van Berkel et al. 2019, p. 119). Currently, the first two types of signals can be set up in the app.

Signal contingent schedules signal randomly within the defined time frame (cf. van Berkel et al. 2019, p. 119). Interval contingent schedules signal at specific times within the time frame in relation to the intervals (cf. van Berkel et al. 2019, p. 119). An additional time constraint can alter this mechanism. This constraint is a specific amount of time that needs to have passed since the last signal in order to avoid two signals in immediate succession (e.g., 9:57 and 10:02). For example, if study participants are asked to report on behavior that occurred within the past half hour, at least 30 min need to pass between two signals to avoid overlap. Signal and interval contingent schedules can be combined so signals are randomized within intervals. This way, researchers can ensure that signals are sufficiently spread throughout the day. Event contingent schedules signal when specific events occur, such as unlocking the device (cf. van Berkel et al. 2019, p. 119). This approach is no longer feasible since Android version 9, as apps no longer have constant access to information such as unlocking and locking times for data protection reasons (cf. Broadcast overview 2022). On iOS, accessing this type of information is not possible either (see section Logging/data donation).

In each interval, it is only possible to fill out an ESM questionnaire once the signal is activated. There are two ways for study participants to access the questionnaire: They can either tap the notification, which opens the app and presents the questionnaire, or they can open the app themselves. The latter option is useful if a participant has removed the notification (either on purpose or by mistake) and still wants to participate during the current interval. If no ESM questionnaire is filled out and submitted by the end of an interval, this process restarts in the next one.

3.3 Logging/data donation

On Android devices, MART can access a log file containing all events that take place on the smartphone. Events are categorized into event types (for more details, see the official Android documentation, cf. Google 2022). They include activities like locking or unlocking the screen, opening or closing apps, and booting or shutting down the device. The log file “contains data from up to two weeks in the past” (Toth and Trifonova 2021, p. 3) up to the time it is accessed. The accessible information includes the event type, a time stamp, and additional information depending on the event type (e.g., app names). Please note that no information on contents received on the smartphone, such as texts or images, is captured.

On iOS devices, events cannot be assessed this way (cf. Apple Inc. 2022). MART still offers a way to gather aggregated phone and app use duration and frequency. This works by giving study participants detailed instructions on accessing the iOS feature Screen Time, describing which information to read there (e.g., daily average use duration and number of pickups during the current week), and asking them to enter these figures into designated fields. This approach is data donation, as the data are not logged passively in the background but manually transferred and curated by participants (cf. Ohme et al. 2021, pp. 294–295).

4 Backend mechanics

MART is configured to retrieve all contents and settings from a WordPress (WP) page. On the WP page, researchers can define and adjust all items and scales for the survey and the ESM questionnaires, as well as the ESM and logging settings. This allows for adjusting content and settings anytime, as the app does not need to be re-compiled from the source code every time something changes. This is also why MART could be provided in the Android and iOS app stores, especially benefiting less tech-savvy study participants (see section Deployment). At the same time, this means that the app requires Internet access to retrieve necessary information (e.g., when accessing an ESM questionnaire). The connection between the app and WP works both ways: survey, ESM, and logging data are transferred from the app to the WP page and are accessible there. Communication between the app and WP is achieved through the WP REST API (cf. WordPress.org n.d.). Technically, MART can be altered to communicate with any other Content Management System (CMS). However, this requires further source code adjustments as it is currently tailored to a WP setup. MART is also provided as a web app that is accessible through a browser. Its main purpose is to enable downloading collected data (survey and ESM answers, logging/data donation data, and information on the operating system used) in separate files (.csv format).

4.1 Deployment

MART is fully functional but not yet customizable without programming knowledge. The source code is available on GitHub (cf. Toth 2023) and will be maintained and updated regularly. Documentation of the procedure for customizing and using the app is also under development.

The author is collaborating with researchers from the University of Bremen to establish an institution that supports MART’s technical and infrastructural advancement. This process also involves developing a system involving a database and account management that 1) lets researchers set up custom studies and 2) allows participants to participate in multiple studies via MART. Providing servers for collected data in accordance with GDPR is also part of the collaboration, so researchers do not need to set up individual servers and look after proper data protection themselves (see Data protection section). MART is available in the Google Play Store [t.b.a.] and the Apple App Store [t.b.a.], which significantly improves accessibility for study participants, as they do not have to manually download and install the app using a .apk file (Android) or another app (iOS, cf. Apple Inc. 2023).

4.2 Data protection

The data collected by MART are anonymized from a technical standpoint. On Android, the logging function only collects the start and end points of phone and app uses. On iOS, it merely asks participants to report an aggregated figure provided by Screen Time. It is therefore impossible to trace the information back to an individual participant. The degree of anonymity of the survey and ESM assessments naturally depends on the items themselves, respectively.

Otherwise, the security of all data transferred with MART depends on the WP setup used, which is left to interested researchers until the collaboration described earlier progresses. While WP offers to use its own servers (cf. WordPress.com 2022), these may not align with the researchers’ data protection requirements and should be checked individually (cf. WordPress.com n.d.). Hosting WP on a custom (local) server (cf. WordPress.org 2022) is possible if this expertise is available. Alternatively, it is possible to use a third-party server hosting service, such as STRATO (cf. STRATO AG 2022) or HostPress (cf. HostPress GmbH 2022), with sufficient data protection measures. These services offer server space, hosting, and administration of a ready-to-use WP page which can then be connected to MART.

5 Prospects

In this article, the author presented MART, an app for Android and iOS devices that enables social science researchers to combine surveys, ESM, and logging/data donation within a single tool. These three data collection methods are used to assess attitudes, behaviors, and most other variables in the social sciences. While many data collection apps have already been developed for research, they each have certain disadvantages—they are either constrained in their functionality, not available for free, only accessible to programmers, or only support a single data collection method. MART aims to combine all relevant data collection methods in an app that can be customized without programming knowledge and qualifies as FOSS.

First and foremost, the development of MART needs to proceed in the way outlined in the section Deployment. It is of utmost priority that creating a WP page on a custom or third-party provider server is automated and simplified as much as possible. As soon as the setup is finished, editing the contents and settings in WP is a rather trivial task that does not differ too much from setting up online surveys with popular providers. To achieve this, a sustainable maintenance and deployment strategy is highly needed and is currently in development.

As a further step, the author aims to add further data collection methods to MART. Like the already available methods, it should be possible to activate and deactivate them individually for each study. One of these additional methods may be the assessment of location (GPS) data (e.g., cf. Bayer et al. 2018) for studies that require precise information on participants’ whereabouts; however, this information is highly sensitive and should only be assessed if absolutely necessary. Another method concerns the assessment of the content received on the device. There are multiple approaches to assessing this, including the frequent, automatic creation of screenshots (cf. Yee 2022). As these contain highly sensitive data, new techniques are currently being developed where the content in such screenshots is identified and encoded into categories on the device—this way, the screenshots can be discarded immediately and participant anonymity is protected (cf. Zerrer et al. 2022). Lastly, eye-tracking allows for precisely measuring participants’ attention to information on the screen, which could be used for complementing self-reported and behavioral data. This method gained increased popularity in social science research over the past years and usually requires complicated and elaborate technical setups (e.g., cf. Ohme et al. 2022, p. 345). However, the increased quality of mobile front-facing cameras has enabled their use as tools for eye-tracking, too (cf. eye square GmbH 2022), which could be implemented in MART in the future.

MART is a mobile application that combines surveys, ESM, and logging/data donation in a single, universal tool that qualifies as FOSS. It does not require programming skills and does not collect unnecessary or sensitive data. It is particularly useful for studies that combine multiple data collection methods but can also be used if only one or two of these methods are needed. It reduces the burden for researchers and study participants by allowing for a seamless transition between different phases of data collection. The implementation of each method is not yet as refined as in applications that only focus on a single method. MART is currently under development and will be available to any researcher to use without requiring programming as soon as possible. To achieve this, the author closely collaborates with other researchers to progress the development and ensure sustainable usability in the future. Feedback is very welcome and will help to improve the final product.

figure a