1 Introduction

Originating from industry as open innovation ecosystems [1,2,3], Living Lab studies have become an increasingly popular domain in research. On the one hand, these Labs offer the unique opportunity to monitor participants’ behavior in real-life settings in real time. On the other hand, they also provide the opportunity to gain iterative feedback and encourage participative user involvement, which can scale up technological and social innovations after the fact by increasing stakeholder involvement and supporting co-creation processes. If the researcher studies global challenges like the climate crisis, the information provided by users is especially important, as the climate crisis is a consequence of human behavior.

To better understand when and why people do or do not act sustainably, behavioral scientists and psychologists have set up Living Labs (LLs) (e.g., as urban sites in buildings) to study diverse topics, such as mobility behavior or household energy usage [4, 5]. Because households and offices are two settings where people spend a great deal of their time, they provide manifold opportunities to engage in sustainable behavior, such as saving energy either to save money or to reduce the personal ecological footprint.

In LLs, these behaviors can be monitored by scientists in several informative ways. For instance, smart technologies such as digital electric meters can be tapped to collect information about the time courses of energy consumption. In the same way, smart plugs can be accessed that monitor the energy consumption of individual devices. In this way, suggestions for when and how often such devices should be used can be generated (e.g., to avoid daily peak times when there is a high energy demand on the grid). Additional information can be gathered via questionnaires (e.g., distributed via smartphones or tablets) to gather information about individual consumption patterns or the user friendliness of certain devices. Based on this feedback, researchers and companies can generate ideas for how to design new and improve existing devices for future use. Moreover, if the investigated LLs are large enough, researchers can compare the performance of participating households, integrate and test gamification elements (e.g., leader boards on which household is most effective in saving energy), and organize co-creation workshops. The latter are used to assess and facilitate user experiences and to develop sustainable, durable and usable technological and social solutions through iterative processes and active user involvement [3, 6]. This non-exhaustive list clearly indicates that there are diverse ways to set up LLs to satisfy the needs of both researchers and participants. However, regarding the promotion of LL studies, the perspective of the participants has not been examined sufficiently.

Which aspects of LLs would make a participation generally attractive to a wide range of people in the first place? The answer is still unclear. After all, participants have to “open their homes” and provide information regularly, sometimes in real-time and by using smart devices, to researchers for a period of several weeks or months. And sharing one’s household behavior requires the participants to display trust towards the researchers, whereas the researchers need to treat sensitive data with particular care and in accordance with national or supranational policy regulations (e.g., the General Data Protection Regulation of the European Union). In general, taking part in a LL study requires the participants to invest effort. If the thresholds (e.g., technical proficiency) are too high, and the individual gains are too low, the participants might not even consider participating for the anticipated duration of a LL study.

In three representative studies, including two exploratory conjoint studies and one confirmatory experimental main study, we tested which aspects might make the participation in LLs attractive. From the results of the exploratory conjoint studies, we deduced aspects that we then tested in the main experimental study along with additional moderating variables. Based on these study results, we provide suggestions regarding what researchers should consider when they plan to conduct a LL study in the future.

2 Summary of Study 1a and 1b: exploratory conjoint studies

Conjoint studies are characterized by the fact that they offer the possibility to test several dimensions and aspects of an object under investigation; these could differ and influence the preference of participants. In our case, the research object is the composition of LLs. We generated relevant dimensions based on our previous projects in the domain of green energy and user participation [7], previous considerations about how to define LLs in the literature [6, 8], digital law perspectives [9], and consultations with other LL organizers using an interview guide (more details can be found in the Supplemental Materials on the Open Science Framework, OSF; https://osf.io/9t6jm/?view_only=f370c90bb22943edb2fd926169b359cb). All dimensions and the aspects that we deemed important are depicted in Fig. 1.

Fig. 1
figure 1

Dimensions, aspects and summary results of the conjoint analysis. Dimensions are in rows and aspects per dimension are in columns; columns S1a and S1b signify, whether evidence (checkmark) for at least a small effect (point estimate AMCE ≥ .05 and p < .05) was found in Study 1a and/or 1b; underlined aspects are more preferred compared to the others; “incentives” was only used in Study 1a and “focus” was only used in Study 1b; details, see https://osf.io/9t6jm/?view_only=f370c90bb22943edb2fd926169b359cb

Full size image

We set up two exploratory online questionnairesFootnote 1 in using LimeSurvey [10] and Pavlovia [11], which we then disseminated via the Talk Online Panel (https://talkonlinepanel.com/at/impressum). In these questionnaires, we introduced participants to the concept of LLs and the dimensions by which the setup could vary in a short video. Afterward, we presented them successively with eight (Study 1a) or five (Study 1b) times two LL profiles and asked them to compare them side by side. The LL profiles consisted of randomly drawn aspects for each of the eight dimensions. Participants indicated their preference for both presented profiles on a seven-point rating scale (ranging from 1 = “not preferred” to 7 = “very much preferred”) and had to choose their preferred LL per page (i.e., the left- or the right-hand profile; see Fig. S1 on the OSF). They then indicated the level of their agreement with the questions on their future orientation (“consideration of future consequences” or CFC, three items) and present orientation (“consideration of immediate consequences” or CIC, three items each), as well as their technical affinity (“affinity for technology interaction” or ATI, four items, [12, 13]),Footnote 2 all measured on a six-point rating scale (see Figure S5 for all items). We drew representative samples from the Austrian population (see Table 1 for descriptive information on Study 1a and 1b). Further information about the scales, methods, and power analysis [15, 16] used can be found on the OSF.

Table 1 Descriptive statistics for all three studies
Full size table

Following the state-of-the-art approaches [17, 18] and using the R packages cjoint [19], lme4 [20] and jmv [21], we analyzed k = 3,696 profiles in Study 1a and k = 2,980 profiles in Study 1b and distilled the most strongly preferred aspects by examining the average marginal component effects (AMCEs). The results show high preference ratings for financial incentives (see Fig. 1; medium and large incentives, 2 and 3, as compared to no incentives, 1, AMCEs > 0.15, p < 0.001). However, most other dimensions did not show large differences in terms of the preferences. Very small effects (AMCEs ~ 0.05) were observed for exchange (3): If no exchange with other users was advertised, this made the LL more attractive than advertising via direct or online exchange (1 and 2). Moreover, a shorter study duration of 1 month maximum was preferred over 6 months and longer (1 as compared to 2 and 3). Finally, participants indicated that they preferred an LL participation from home instead from a modern apartment (location, 1 as compared to 2). Effects of the remaining dimensions were too close to zero to be relevant.

3 Study 2: confirmatory experimental study

In Study 2, we tested the consistency of the surprising finding from Study 1a that promoting exchange with other users would lead to a lower attractiveness of LLs. To maximize the exchange effect in this experimental setting, we combined the variables exchange (with other users) with guided learning (through researchers), thereby emphasizing the fact that participants could either explore the LL individually or by interacting with other users and researchers. Moreover, instead of introducing LLs in videos, we modified the focus condition from Study 1b by using digital flyers that were then disseminated to theoretically advertise the participation in LLs.Footnote 3

Although neither exchange nor focus showed effects in Study 1b, we deemed these aspects to be relevant with regard to the overarching research question about factors that influence LL attractiveness, which would be crucial for promotion strategies.

3.1 Materials and methods

For the variable focus, we created three flyers: The first advertised technical aspects of the Living Lab (tech), the second advertised sustainable energy usage in the Living Lab (green), and the third advertised social participation in fair energy systems (social). Using LimeSurvey [9], we presented one of the flyers to participants as a between-participants condition. Exchange with other users and researchers was then varied as a within-participants condition: The order of two flyers (which had a similar focus) was randomized and either contained information about interactions with other participants and researchers or only about individual exploration. We again asked the participants to assess the attractiveness of the LL (dv1) and describe whether they would consider participating (choice, dv2), and established the overall attractiveness of the flyer as a control variable (all measured on a seven-point scale, ranging from 1 = “not at all” to 7 = “very much”). In the period between presenting these flyers, we again asked participants to describe their future and present orientations, as well as their technical affinity.

3.2 Results

With data from N = 524 participants, we conducted the 2 (within, exchange: none, much) × 3 (between, focus: tech [n = 195], green [n = 155] and social [n = 174]) mixed ANOVA with attractiveness of the Living Lab and whether they would consider participating (choice) as dependent variables. The results are depicted in Fig. 2 and show that neither factor made a difference: Regardless of the flyer design or—with respect to the hypothesis—an emphasis being placed on the exchange with others, participants rated the attractiveness with M = 4.96 (SD = 1.29) and choice with M = 4.46 (SD = 1.68).

Fig. 2
figure 2

Results of the confirmatory analysis. Estimates represent means with 95% Cis; scatter represents participants in groups

Full size image

We also measured the participants’ levels of technical affinity and their future and present orientations, which we added independently to the models. The former two, affinity (dv1: F(2, 515) = 12.04, p < 0.001, ηp2 = 0.05; dv2: F(2, 515) = 22.68, p < 0.001, ηp2 = 0.08) and future orientation (dv1: F(2, 515) = 33.57, p < 0.001, ηp2 = 0.12; dv2: F(2, 515) = 19.70, p < 0.001, ηp2 = 0.07) showed main effects across both dependent variables, with people with high levels of affinity and a strong future orientation perceiving the LLs as more attractive than people with low levels of affinity and a weaker future orientation, although these mean differences were mostly on the level of 0.5 scale points. Detailed results can be found in the supplemental document (https://osf.io/9t6jm/?view_only=f370c90bb22943edb2fd926169b359cb, Tables S5–S16).

4 Discussion

As LLs have become increasingly attractive for researchers to receive information about people’s energy consumption in real-life settings, we set out to test, which factors might make the participation in such a study attractive. After all, insights gained in LL studies may help encourage sustainable and pro-environmental behavior.

However, the results of our three studies are sobering: Two exploratory conjoint studies showed that LL study features such as a minimum level of interaction with other participants, a short duration (less than 1 month), and the possibility to participate from home (rather than in a digitalized apartment) could boost the attractiveness of the LL, somewhat encouraging individuals to participate, while financial incentives could boost their motivation a lot. However, these results—with the exception of the financial dimensions—are likely unstable in real advertisements of LLs, as we showed in Study 2; here, we could not replicate the somewhat surprising effect that having little exchange with other users is attractive. Moreover, although the flyers were meant to increase the effect of the focus dimension, our results show that it did not make a difference whether technical, green, or social aspects were emphasized as a motivation to participate in a LL study. Instead, we found some evidence that people with a strong future orientation in their decision-making and with a high affinity towards technology preferred to participate in LL with an energy-usage focus more than people who scored low on these scales.

A crucial limitation of our studies is that they simulated one-shot and low-cost recruitment strategies: Participants assessed the attractiveness of LLs based on information in videos or flyers and knew that they would not actually be recruited for such a study. Hence, their overall slightly positive attractiveness ratings may just represent a response bias that reflects a neutral position. Next, although we carefully chose the dimensions described, it is possible that our selection of aspects may have negatively biased the results: For instance, the study duration dimension contained one short duration (1 month) and two longer durations of 6 months and more, without other aspect options in between. Additionally, the effects of the psychological predictors future orientation and technology affinity might point to a larger problem: potentially, the recruitment for LLs studies could result in a selection bias independent of the advertisement strategies. If mainly these two groups of people feel addressed, the inferences drawn from LL studies might also be limited to these groups.

5 Suggestions and future directions

LL studies are a promising means of integrating users into process of technology advancements via participative structures, as well as a means of helping researchers to get unique insights into household energy consumption. Taking advantage of these two aspects of LLs, suggestions can be made for how to improve new and smart technology that helps people adopt more energy-efficient and sustainable behavior. However, our current study findings imply that recruitment for such a study could require an investment of effort, and especially if it is supposed to be representative of the whole population (instead of appealing to future-oriented and tech-savvy people only).

Researchers who plan to conduct an LL study in the future are advised to devote both a lot of time and financial resources during their study planning for recruitment. We suggest i) that researchers plan well ahead of beginning their projects in consultation with local authorities and private organizations in order to discuss how such projects could be advertised (e.g., newspaper, social media adds, at public places off campus). Personal contact and the exchange of experiences with other researchers who have conducted LL studies before may also help them to identify useful strategies. Additionally, we suggest ii) to have a diverse set of user groups partake in the early design stages of an LL study. Following this procedure, although it seems more laborious at face value, may enable the researchers to identify user-specific needs early on and motivate them to be and remain a part of the project throughout its duration [e.g., 22, 23]. As financial compensation was identified as the most prominent motivating factor, we also suggest that researchers provide (financial) incentives, which might vary according to the socio-demographic group and study duration (e.g., milestone incentives to counteract a dropout and attrition rate). Not all of these measures will be actionable in all contexts, but careful planning might increase the participation rate.

6 Conclusion

Overall, we studied the attractiveness of LLs utilizing three large samples. We found very limited evidence for a few factors that could be used when advertising LL studies, including a shorter study duration and the possibility to participate from home. Prominently, financial incentives stood out as the most influential factor. We suggest that researchers who want to conduct an LL study should actively plan to offer incentives and choose advertising strategies that go beyond low-cost recruitment strategies.