To demonstrate the use of formr, we chose three case studies that the authors told us would have been difficult or impossible to realize using other open tools, short of programming customized studies or foregoing automation. All three led to articles that were published in the Journal of Personality and Social Psychology.
A dyadic study with feedback for both partners
The formr software can be used to implement dyadic studies, such as studies involving both partners of a couple or best-friend dyads. In a recent multistudy investigation, Wurst et al. (2017) examined the effects of narcissism in early and later phases of romantic relationships. In particular, in their Sample O, they investigated actor and partner effects of narcissism using data from 272 committed couples. To collect these data, two separate runs—one for the first partner and one for the second partner—were implemented. Participants were recruited via email lists, online social networks, snowball sampling, and advertisements in lectures. Participants entered the first run and filled out a number of questions about their personality and their relationship. At the end of the surveys in the first run, participants were asked to share a personalized link with their partner in order to recruit them into the second run (the contents of the second run were identical to those of the first). This personalized link transmitted the partner’s email address, and optionally their personality data, although it would now also be possible to transmit such information invisibly using the experimental formr API.Footnote 18 In addition, participants had to indicate whether they would be comfortable with their personality data being used for the dyadic feedback. After their partners had completed the second run, if they had also agreed to their data being used for the dyadic feedback, both partners received a dyadic personality profile.
A longitudinal study with a social network component and other ratings
In the Göttingen Mate Choice Study (GMCS)—a multiwave investigation into partner preferences, relationship transitions, and relationship development—Gerlach, Arslan, Schultze, Reinhard, and Penke (2019) followed up on a large sample of singles. Using longitudinal data from the first two waves of the study, they investigated whether singles’ preferences predicted the characteristics of later partners. As such, the study’s prospective design was unique in the study of partner preferences. However, along with following up with participants longitudinally, the GMCS also incorporated further innovative features, such as a social network component and the assessment of participants and their partners through their peers.
In Wave 1, participants were screened to include only singles on the lookout for a potential partner. These participants initially answered some questions about their demographics, personality, and ideal partner preferences. They were then asked to provide a list of all potential romantic partners and to rate each on several attributes. Five months later, they received an email inviting them to the second wave of the study. In Wave 2, the authors reiterated the questions about ideal partner preferences and followed up on participants’ relationship experiences since Wave 1. These experiences included current and past partners, with both individuals from the network of potential partners and new people that participants might have met later. Crucially, by supplying the list of names from the previous wave as choice options in Wave 2, the authors could link the partners recruited from the network of potential partners with their Wave 1 ratings. In all cases, the authors obtained ratings on the same attributes for the chosen partners. Eleven months after the start of the study, the authors initiated Wave 3, which asked the individuals who had taken part in both previous waves to have their partners rate themselves and to have them rated by peers. This was implemented by setting up two additional studies (one for partners and one for peers), for which participants received links that included their participant code. Peers and partners who received this link via email were then invited to rate the partner and the original participant on the aforementioned attributes.
A daily diary study with a social network component
The Daily Habits and Sexuality Study (Arslan, Jünger, Gerlach, Ostner, & Penke, 2016) collected data from 1,345 women over a period of 70 days. The women were first screened on a variety of demographic criteria. The main objective was to assess behavioral and psychological changes across the menstrual cycle. Thus, depending on whether the women were predicted to ovulate regularly according to a set of demographic and health criteria, they were offered either a monetary reward or entry in a lottery. Both rewards were calculated on the basis of an algorithm that took into account the frequency of participation and specifically rewarded a lack of large gaps in participation in the diary with a bonus. After being told which reward scheme they would receive, participants answered personality questionnaires and closed their browser.
Over the following 70 days, participants received a daily email invitation that told them how many days they had completed, how much money they had earned, and whether they had skipped the last day. If they had given their mobile phone number, they also received a text message reminder if they had not reacted to the previous day’s email or were in danger of losing their bonus.
In the diary, participants answered some items daily, and other items were randomly shown only on a subset of days, to keep the time needed to fill out the diary under 5 min per day while maintaining construct breadth (Revelle et al., 2017) and reducing rote responses.
Each day, single women noted who they had interacted with or thought about on that day (using first names or nicknames). The names of the interaction partners were cataloged using R. For the ten most mentioned names that were noted at least three times, women were asked to indicate whether these names belonged to men who were not their relatives. If they did, women answered a number of questions related to their attraction toward these men. This loop in formr continued until at least four unrelated men or at least ten persons in total had been assessed (assuming that enough names were mentioned in the diary). Unlike a standard social network questionnaire, which tries to elicit the names of interaction partners by relying on a participant’s memory and ability to generate names, this method made it possible to focus on the people who actually were the most commonly reported interaction partners in the diary period.
Participants then filled out a general follow-up questionnaire. In the end, they received automated, graphical, interactive feedback that included not only standard graphs, such as personality feedback, but also visual representations of how their mood, desire, and stress level varied across the menstrual cycle (see Fig. 4) and how they spent their time on different days (see Fig. 5). Scatterplots with fit lines were generated in order to let them examine whether there was any correlation between their alcohol consumption on the previous day and their sleep quality and mood the next day. They could also view the changes in their mood throughout the diary in an interactive display that allowed them to consult their notes on that day, to see why some days might have differed from others.
On average, women responded on 43 days during the diary period, and only 19% did not complete the follow-up (among those who were paid per response, the average response rate was 49 days, with 10% not completing the follow-up). Although the participants were not formally surveyed on this, online interactions indicated that the feedback was generally well received. The complete study is reproducible from documentation on the Open Science Framework and can be imported and changed (Arslan et al., 2016). An earlier, similar study implemented in an older version of formr was recently published and is described in more detail in Arslan, Schilling, Gerlach, and Penke (2018).
Other possible designs
The three case studies show how formr has been commonly used in ways that make full use of its feature set. However, we believe that a key strength of formr is that it can be used in ways that we did not plan for. For example, it would be possible to design an event-contingent monitoring study in which alerts are sent to users whenever a sentiment analysis shows an uptick in the use of emotion words on a social media platform. This would be possible because formr’s R support includes R packages that enable automated calls to the Twitter API and sentiment analysis. Similar applications could respond to weather events or events gleaned from screen-time monitoring apps. Contingent events could also be a partner’s or friend’s behavior in a dyadic design, evaluated using the formr API. A number of users have also used formr in the lab to obtain stimulus ratings and simple experiments, because they found randomization to be easy in formr. Some have even used formr as a vehicle for reaction-time-based experiments, by extending it using JavaScript.