1 Introduction

Norway is a seafaring nation and has experienced throughout history many memorable maritime emergencies and crises. Recent examples are the 1980 capsizing of the semi-submersible drilling rig ALEXANDER L. KIELLAND (Norwegian Official Report 1981), the arson onboard the car and passenger ferry SCANDINAVIAN STAR in 1990 (Norwegian Official Report 1991), and the environmental damage caused by the oil leak from container vessel M/V GODAFOSS in 2011 (Accident Investigation Board Norway 2011). Despite different types of incidents, all evaluation reports have pointed to the same problem—insufficient cross-sector collaboration (Norwegian Official Report 1981, 1991; Accident Investigation Board Norway 2011). Maritime crises in certain respects can be considered more challenging than on-land incidents. For example, distance to first responders is often longer, weather conditions can impede rescue operations more significantly, aid and recovery resources are often more highly specialized, and interagency coordination in pursuit of a common goal is often difficult to achieve. Insufficient collaboration in emergencies, however, is generic issue not just a maritime problem.

The official evaluation following the 2011 terrorist attacks on Norway stated, among other conclusions, that the response had been insufficient, and that cross-sector collaboration efforts in particular had been inadequate (Norwegian Official Report 2012). In response to the attacks, in 2012 the Stoltenberg government introduced collaboration as a fourth national emergency preparedness principle together with the existing principles of responsibility, equality, and proximity. According to the White Paper (Norwegian Ministry of Justice and Public Security 2012) that introduced the principle, collaboration was to be developed through a stronger emphasis on collaboration exercises. In crisis management, there is a widespread assumption that cross-sector collaboration exercises test and develop, among others, preparedness and cross-sector team integration (Rutty and Rutty 2012). The problem, however, is that crisis collaboration exercises both on land (Berlin and Carlström 2008, 2009, 2015) and at sea (Kim 2013, 2014; Kristiansen et al. 2017; Sørensen 2017; Magnussen et al. 2018; Sørensen, Carlström et al. 2018; Sørensen, Magnussen et al. 2018) tend to produce results with limited usefulness in real crisis work. Why this perceived situation exists is debated, but cited reasons include insufficient attention to variation (Borell and Eriksson 2013), insufficient focus on learning aspects (Berlin and Carlström 2015), dominance of mechanistic behavior (Berlin and Carlström 2013), and too much dependence on standardization (Kim 2013). Despite these legitimate criticisms, a focus on the development of varied scenarios that gradually improve participants’ collaborative behavior is an important organizational task (Sørensen, Magnussen et al. 2018).

We believe, however, that lack of a research-based knowledge, together with an absent national maritime collaboration exercise framework, makes it difficult for maritime exercise organizers to achieve and facilitate adequate collaboration development for all. This is especially true when many of the participants belong to the exercise planning organization, and thus may be familiar with internal exercise planning procedures and scope traditions. To test the hypothesis that belonging to an exercise planning organization has an effect on the participant’s perceived levels of collaboration (C), learning (L), and utility (U), the purpose of this nonexperimental, survey-based study was to measure the difference in perceived levels of CLU between participants belonging to the exercise planning organizations and participants belonging to other participating groups. Although collaboration is an integrated part of many countries’ emergency frameworks, Norway is, to our knowledge, the only country in Europe that has collaboration as an official national principle. Because the amount of international literature on the perceived effects of cross-sector collaboration exercises is scare, this study can make an important theoretical contribution to an emerging field. A focus on collaboration exercises both on land and at sea has become more prominent following the 2011 terrorist attacks. An increased knowledge about the effects of these exercises will help future exercise planners design exercises in a way that leads to increased perceived levels of learning and utility. Increased knowledge about maritime collaboration exercises will help maritime exercise designers concentrate on possible maritime problem areas such as longer distances, more limited access to equipment and personnel, and the use of maritime communication channels that differ from those on land. The study also should be relevant to policymakers, since it points out the importance of designing exercises in a way that ensures optimal learning and utility value.

2 Background

In crisis work, the ability to mobilize resources and to facilitate cross-sector collaboration is an important success factor (Lu and Xue 2016). Collaboration is a horizontal process where stakeholders, based on the assumptions that the benefits are higher than the costs (Andersson et al. 2014) and prestige seeking is limited (minimal desire for personal gain), engage in partnerships to resolve complex situations (Murphy et al. 2015). Organizationally, collaboration became a popular working trend in the late 1980s, following a period in which a strong competitive focus dominated (Axelsson 2000). Today collaboration is described as both democratizing and efficiency promoting (Barker 1993) and is dominant in both public and private sectors (Sullivan et al. 2012). Collaboration is also viewed as a solution to sub-optimal task distribution and regulatory fragmentation (Drucker 2007). Not engaging in cross-sector collaboration may result in reduced efficiency, flexibility, and resilience in times of crisis (Jung and Song 2015), as well as make it harder for managers to impose order and deal with hostile consequences (Salman Sawalha 2014).

2.1 Collaboration Exercise Learning

Learning about collaboration during exercises has multiple origins. The concept originates in Johan Stein’s (1997) first- and second-order learning theories, which can be linked to Klabbers’ (1999) learning model and complimented with Argyris’ (1976) theory on organizational learning as well as Argyris and Schön (1978) theory on single-loop and double-loop learning. From a crisis management perspective, first-order learning occurs when individual or groups acquire new knowledge but are unable to internalize learning completely or apply their new knowledge to real-life scenarios. In contrast, second-order learning occurs when partakers show the ability to transform and apply new knowledge as a tool to solve or improve future situations (Berlin and Carlström 2015). The goal of learning is not only to gain new knowledge (Sommer and Njå 2012); it also represents development (Sommer et al. 2013). The concept of learning is argued about among scholars, but two acknowledged understandings concern individual cognitive learning and sociocultural learning (Sommer and Njå 2012). Cognitive learning refers to the individual’s ability to acquire and reflect on outside information (Bahri and Corebima 2015), whereas sociocultural learning focuses on interpersonal relations (Sommer and Njå 2012). To develop both cognitive and sociocultural learning, cross-sector exercises must clearly encourage and facilitate collaboration behavior and have a clearly stated collaborative purpose (Andersson et al. 2013). In an exercise setting, there are several ways to develop collaboration learning—for example, through limited exercise scopes, limited waiting periods, and clear collaboration directives before and during the exercise (Berlin and Carlström 2015).

2.2 Collaboration Exercise Usefulness

Exercise utility refers to the improvement of existing cross-sectoral capacities to handle emergencies or crises (Andersson et al. 2014). But the existing literature points to the tendency that both land-based and sea-based cross-sector collaboration exercises produce limited perceived levels of usefulness (Berlin and Carlström 2009, 2015; Kim 2013, 2014; Kristiansen et al. 2017; Sørensen 2017; Magnussen et al. 2018; Sørensen, Carlström et al. 2018; Sørensen, Magnussen et al. 2018). The goal of cross-sector collaboration exercises should therefore not always be to practice something pre planned, but rather to focus on collaboration development (Borell and Eriksson 2013). By coming together, the different participants and sectors are given the opportunity to engage in joint discussions, make mistakes, and test alternative strategies. Coming together also allows for the creation of formal and informal relations, as well as establishes a sense of security and structure across sectors (Andersson et al. 2013). Torgersen (2015) pointed out that for cross-sector collaboration to be effective, exercise organizers and planners must arrange for a basic understanding of organizational culture and adaption among stakeholders. Such understanding may influence both actual and perceived learning outcomes internally and across sectors. Essential to this objective is the understanding and acceptance of each other’s governance model and organizational culture, which again creates an arena where stakeholders can adapt to each other during response efforts, regardless of which organization has been appointed to lead exercise or response activities. Stakeholders must also be willing to engage in a collaborative process, where basic levels of trust and commitment must exist (Kauser and Shaw 2004). The concept of trust is debated among scholars, but most scholars and practitioners agree that, despite individual variances, there is a relationship between trust and collaboration (Ross and LaCroix 1996; Dirks and Ferrin 2002), and that trust serves as one of the main prerequisites for effective collaboration (Kouzes and Posner 2003). Ekman (2012) found that when it came to trusting, professional familiarity across boundaries was more important than cultural perspectives. Ekman’s findings were confirmed by Pramanik et al. (2015), who found in a Swedish experimental collaboration study that there was a strong group bias among participants and that these partakers tended to more easily trust members of organizations that they knew beforehand and with whom could readily identify. Group bias has also been found during exercise planning processes, and this has resulted in unnecessary ad hoc processes and sector-specific exercise-script development (Paton et al. 1998). Kim (2013) has argued that when participants became too focused on individual, sector-specific exercise goals rather than collaboration development, the group lost sight of important elements of collaboration, which again leads to a decreased utility in real crisis response.

3 Method

This study employs a quantitative, nonexperimental, survey-based methodology. A survey design was chosen over other methodologies and designs as it was deemed most suitable for large population groups and allowed for the desired variable testing through statistical procedures (McCusker and Gunaydin 2015). Our study focuses on one exercise—SKAGERRAK CHEMICAL OILSPILL EXERCISE (SCOPE), a 2017 full-scale maritime chemical oil-spill pollution collaboration exercise based on assigned project funding from the EU Commission Directorate-General Humanitarian Aid and Civil Protection. The exercise was coordinated by the Norwegian Coastal Administration, NCA (2017). The goal was to train first responders in crisis management and collaboration skills. The exercise occurred off the coast of Norway. It included partners and participants from Norway, Denmark, Sweden, Iceland, and Germany. The exercise was chosen as a study object because it was a full-scale, cross-sector and cross-border, collaboration exercise with both national and international maritime participants. Informed consent for our study was given by the exercise planning and organizer organization. The study also was a part of an ongoing Norwegian research project on Emergency Management and Interaction during Crises—in Maritime Context, approved by the NSD—Norwegian Centre for Research DataFootnote 1 (Ref. 44815). NSD is the Data Protection Official for Research for all the Norwegian universities, university colleges, and several hospitals and research institutes. The project was carried out according to NSD standards and requirements. An electronic survey was used to collect data and measure the exercise participants’ perceived levels of collaboration, learning, and utility. The survey was designed in QuestBack and distributed jointly to all operative 600 participants through a hyperlink by the exercise planning and organizing organization. In QuestBack, the “hide identity” option was selected, which according to the QuestBack (2018) Security Statement ensures that the IP addresses, e-mail addresses, and browsers type of participants were not identified. Data were imported and analyzed in Statistical Packages for the Social Sciences (SPSS) version 24.0. The target population was all operative, exercise participants; exercise designers, the exercise directing staff, and other strategic personnel were excluded. The sample population consisted of operatives associated with the exercise organizer organization and others belonging to other public and nongovernmental emergency organizations. The data collection instrument was the “Collaboration, Learning, and Utility Scale” (CLU-scale), which is a validated Swedish-designed instrument to measure the effects of collaboration exercises (Berlin and Carlström 2015). In addition to answering questions related to perceived levels of collaboration, learning, and utility, the participants were asked to provide demographic data about their gender, age, professional affiliation, number of years of professional experience, and number of collaboration exercises attended prior to participation in SCOPE 2017.

To test whether organization affiliation had an impact on perceived levels of collaboration, learning, and utility, all the respondents, regardless of age, gender, or professional experience, were divided into two groups: (1) those who belonged to the exercise planning and organizing organization (NCA); and (2) those who did not (“others”). To test whether there was a statistical difference when it came to perceived levels of collaboration, learning, and utility values, the individual mean scores of each group on items within the collaboration, learning, and usefulness (CLU) dimensions were first calculated, and then item scores were aggregated for each dimension by averaging them within each. To test the hypothesis that there was a statistically significant mean difference between the exercise organizer group and the “others” group when it came to perceived collaboration, learning, and utility, an independent sample t test was then performed. The alpha significance level was 0.05 (Cohen 1988). The CLU-scale consists of an array of dimensions and items, which are displayed in Table 1. A five point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) was applied to quantify the responses.

Table 1 Collaboration, Learning, and Utility scale (CLU-scale)
Full size table

4 Results

This section reviews the participant responses to the online CLU survey. An overview of sample distributions and demographics is given. Quantitative findings are reported, and the mean (M) and standard deviation (SD) values also are listed when appropriate.

4.1 Sample Distribution and Demographics

Ninety persons participated in the study (response rate: 16%). Six persons were excluded from the dataset because, although they had agreed to participate in the study, they only responded to a few demographic questions and failed to answer any of the CLU questions. Of the 84 remaining sample participants, 59 were males and 25 were females. Forty-one belonged to the exercise-organizer organization and 43 belonged to other participating organizations. Of the five categorized age groups covering ages from 18 to 55 + , all age groups were represented, except the 18–24 group (M = 3.80, SD = 0.91). The number of years of professional experience was divided into five groups and ranged from the 0-5 group to the 21 + group (M = 3.33, SD = 1.54). The number of collaboration exercises attended prior to this one was also divided into five groups and ranged from the 0–1 group to the 11 + with a mean in the 2–4 group (M = 2.82, SD = 1.45).

4.2 Collaboration

When asked whether the exercise focused on collaboration, the mean sum of participants who belonged to the exercise organizer organization was 4.37 (SD = 0.86), while the mean of the “others” was 4.42 (SD = 0.95). In response to questions about whether the exercise participants perceived that there had been sufficient forms of discussions provided during and immediately after the exercise, the organizer group exhibited a mean of 2.78 (SD = 1.27) while the “others” had a calculated mean of 3.33 (SD = 1.28). Questions about the opportunities provided by the exercise to improvise and try out new strategies, in the organizer group showed a mean of 3.68 (SD = 1.03), while “others” exhibited a mean of 3.88 (SD = 1.05). The item “collaboration was initiated immediately” displayed an organizer group mean of 3.80 (SD = 1.07) and a 3.79 (SD = 1.28) among “others” participants. When responding to whether participants had performed well-known activities, the organizer group displayed a mean of 4.24 (SD = 1.09) while the “others” group displayed a mean of 4.47 (SD = 0.90). When calculating the mean associated with the item “personnel in need of exercises participated,” the organizer group came out with a mean of 4.71 (SD = 0.64) and “others” with a mean of 4.35 (SD = 0.89). The mean among organizers to the item “clear collaboration instructions were presented” was 3.27 (SD = 1.39) and 3.81 (SD = 1.27) among “others.” To the perceived participant experience of whether their points of view had been regarded, the organizer group presented a calculated mean of 4.10 (SD = 0.99) and “others” a mean of 4.33 (SD = 0.86). The calculated mean of the items of the collaboration dimension was 3.86 (SD = 0.64) for the exercise organizer group and 4.04 (SD = 0.63) for the “others” group.

4.3 Learning

Perceived experience of whether they had learned something new during the exercise showed a calculated mean of 3.95 (SD = 1.37) among the participants from the exercise organizer organization and a calculated mean of 4.63 (SD = 0.53) from participants from other organizations. When it came to whether the participants perceived that they had learned something about the organizational aspects of other participating organizations, the organizer group had a calculated mean of 3.78 (SD = 1.33) and the “others” a mean of 3.84 (SD = 0.97). Related to the issue of whether they had learned something about the communication patterns of collaborating organizations, the organizer group presented a participant mean of 3.66 (SD = 1.37) and the “others” group a mean of 3.58 (SD = 1.07). The exercise organizer group displayed a calculated mean of 3.24 (SD = 1.28) to the item of whether they had learned about the way collaborating organizations prioritize their activities, while the “others” group a mean of 3.44 (SD = 0.98). To the perceived participant experience of whether they had learned new concepts and abbreviations belonging to collaborating organizations, the organizer group displayed a mean of 3.29 (SD = 1.23) and the “others” presented a mean of 3.49 (SD = 1.03). The calculated mean of the items of the learning dimension was 3.58 (SD = 1.13) for the exercise organizer group and 3.79 (SD = 0.72) for other participants.

4.4 Utility

Perceived experience of whether the exercise had been useful to real-life activities during actual emergency work displayed a calculated mean of 4.26 (SD = 1.12) among the participants from the organizing organization and a mean of 4.70 (SD = 0.59) among other participants. For the item “based on what I learned, the exercise were useful to command officers,” the organizer group had a calculated mean of 3.17 (SD = 1.04) and the “others” group a mean of 3.65 (SD = 1.23). When asked whether the exercise were useful to ordinary operative staff (not command officers), the organizer group responded with a mean of 3.17 (SD = 0.91) and the “others” with a mean of 3.30 (SD = 1.05). Finally, the participants were asked whether they perceived, based on what they had learned, that the experiences from the exercise will have an impact on their daily work. Here the exercise organizer group displayed a calculated mean of 2.88 (SD = 1.24) and the “others” group a mean of 3.28 (SD = 1.16). The calculated mean of the items of the utility dimension was 3.37 (SD = 0.64) for the exercise organizer group and 3.73 (SD = 0.66) for other participants.

4.5 T-test

To test the hypothesis that there was a statistically significant mean difference between the exercise organizer group (N = 41) and the “others” group (N = 43) when it came to perceived collaboration, learning, and utility, an independent sample t-test was performed. A visual inspection of collaboration, learning, and utility histograms, normal Q–Q plots, showed that the output of each group was approximately normally distributed with skewness values < 2.0 and kurtosis < 9.0 (Schmidler 2010). Assumption of homogeneity was tested and satisfied using Levene’s F test for quality of variance. The exercise organizer group displayed a mean collaboration value of 3.86 (SD = 0.64) and the “others” group a mean of 4.04 (SD = 0.63). The independent sample t-test was associated with a nonstatistical effect (t = − 1.27, p = 0.20), which indicated that the mean score between the groups was not significantly different. When it came to learning, the exercise organizer group displayed a mean collaboration value of 3.58 (SD = 1.13) and the “others” group a mean of 3.79 (SD = 0.72). The independent sample t-test was also found nonstatistically significant (t = − 1.00, p = 0.31), which meant that the mean learning scores between the two groups was not significantly different. The third dimension, utility, had a calculated mean of 3.37 (SD = 0.64) among the participants from the organizing organization, and a mean of 3.73 (SD = 0.66) among other participants. Compared to collaboration and learning, the t-test did here find a statistical significance of 0.01 (t = − 2.47), which meant that the mean between the two groups was significantly different. Table 2 gives an overview of the mean results and t-values for the two groups and their answers to CLU.

Table 2 Mean results and t-values of the exercise organizer and non-organizer groups for collaboration, learning, and utility
Full size table

5 Discussion

The 2012 change to Norwegian emergency legislation, which added collaboration as a fourth national emergency preparedness principle, has put a new responsibility on organizers of emergency preparedness exercises. As an effect of the new legislation, an expectation has emerged that training exercises will be carried out regularly. These exercises are assumed to incorporate not only command and control, technology, emergency plans, but also are extended to include enhanced collaboration between agencies at all levels. Studies of collaboration exercises and real emergency work in Scandinavian countries have revealed a lack of collaboration efforts by differentiating the actions undertaken in specific organizational operations such as health care, firefighting, and security (Berlin and Carlström 2009, 2013, 2015; Kristiansen et al. 2017; Sørensen 2017; Magnussen et al. 2018; Sørensen, Magnussen et al. 2018). Because Norway was the first country to include collaboration as a priority in its emergency planning, it has had to pioneer a focused effort to achieve interorganizational consensus, to develop elaborate common strategies, and to use all available resources to stabilize and establish normal order (Norwegian Ministry of Justice and Public Security 2012). Despite implementation of collaboration as an official principle, the results in this study and similar maritime research (Magnussen et al. 2018; Sørensen 2017; Sørensen, Carlström et al. 2018; Sørensen, Magnussen et al. 2018) indicate that the participant’s perceived mean outcomes of CLU are similar to those found in existing international land-based exercise studies (Berlin and Carlström 2009, 2015). Studies on the effects of Norwegian land-based exercises, however, still in large part must be performed and should be considered a gap in the literature.

In accordance with the effort to improve collaboration, the results of this maritime study imply an ambition among the organizers of exercises to achieve the new legislated standard of collaboration. This ambition seems to be higher than the ability to construct an exercise focused on collaboration. The mean CLU values determined in our study show that members of the exercise planning organization reported overall lower mean scores than the participants from the participating organizations. This indicates that exercise design and implementation were not optimal to ensure equal CLU goal completion among all the participants. The overall figures showed a mean of the collaboration variable of 3.86 for the exercise organizer group and 4.04 for others, learning showed 3.58 versus 3.79 and utility 3.37 versus 3.73.

One item in the collaboration dimension—whether the exercise participants perceived that there had been sufficient forms of discussions provided during and immediately after the exercise—differed notably in means (organizers 2.78 and others 3.33). Tentatively, the figures can indicate that an intensive preparation phase resulted in exhaustion among organizers. Their ambition to participate in the discussions might have been higher than they were able to manage. The issue of whether their points of view had been regarded pointed in a similar direction. The organizer group presented a calculated mean of 4.10 and others 4.33. Obviously, the visitors were rather satisfied with the efforts of the organizers to arrange an effective exercise.

The most remarkable difference between organizers and others was in the dimension of utility. The visitor group did, to a relatively high degree, find the exercise useful and applicable to real-life events during actual emergency work (4.70), while organizers were more cautious but still positive (4.26). Compared to other recently studied Scandinavian maritime collaboration exercises, this exercise revealed a positive development. In a study of three 2016 exercises, Sørensen (2017) found a calculated CLU mean of 3.75. A study of Exercise NORD 2016 revealed an overall mean of 3.09 (Magnussen et al. 2018), and a study of a 2017 joint Norwegian-Swedish maritime Search and Rescue collaboration exercise displayed a mean of 3.66 (Sørensen, Magnussen et al. 2018). We consider this development positive, but the results still show the need for further focus on collaboration learning and development. The results in this study suggest that among organizers the ambition to accomplish collaboration was stronger than the outcome. However, the external participants found that the exercise was truly a collaboration exercise. They also found it instructive and useful when applied to real life events. In contrast to Kim’s (2013) findings, the SCOPE 2017 exercise did not seem to only focus on sector-specific exercise-script controlled elements, there were also collaboration elements included, but the results indicate that there is still room for improvement. Self-criticism is an advantageous character trait when new legislation is launched. The opposite, lack of insight into one’s own shortcomings, will not provide improvement.

6 Conclusion

This study tested whether there was a difference in perceived exercise effect among regular participants and participants belonging to the exercise planning organization. Findings indicate that levels of collaboration, learning, and utility were higher among other participants than among those belonging to the exercise planning organizations. The results of the study also indicate that new legislation can have some effect on crisis organizations, despite lacking national collaboration exercise templates and waiting procedures on how such exercises should be evaluated in terms of goal achievements. Ekman’s 2012 study revealed that participants in collaboration exercises tend to trust more readily members of organizations that they previously knew; they could identify with each other better if exercise organizers and participants focus on constructing collaboration elements in exercises. Based on the optimistic assumption that collaboration can occur, there is good reason to continue the development of collaboration exercises as well as promote collaboration during real-life crisis events in Norway.

From the perspective of maritime collaboration exercise planners, the results indicate the need for a redesign of exercise scripts in a way that facilitates a participant’s perceived levels of CLU regardless of organizational affiliation. We recommend an inclusion and facilitation of surprised unforeseen events that amplifies already known maritime challenge areas such as distance and communication, as these will force participants to communicate, share information, and improvise across sectors. We further recommend that a national maritime collaboration exercise framework should be developed and adopted. A practical implication is a recommendation to evaluate all exercises to secure the outcome regarding collaboration skills. A regularly used questionnaire such as the CLU instrument can register the long-term value of exercises. Such a procedure could be of great help in order to adjust to new regulations and improve their outcomes.

Finally, we recommend that this study’s result also form the baseline for further studies on the effects of collaboration exercises. Especially do we recommend that similar studies are performed on Norwegian land-based exercises for comparison purposes. Although the results from Norwegian maritime exercises are quite similar to those found in the existing land-based international literature, the lack of research should be considered a gap, and the maritime results should not be generalized uncritically.

This study was limited in scope because data were collected from only one maritime collaboration exercise. By not performing face-to-face data collection, our survey methodology could have missed capturing verbal and nonverbal cues and other verbal signals, as well as conducting more accurate screening. Despite three reminder messages, a third limitation was the overall low response rate (16%). Unfortunately, this seems to be an increasing trend in Norway. Already back in 2013, the Research Council of Norway (2013) noted in an article published online that the numbers of respondents willing to respond to surveys had dropped sharply, and that the average response rate had gone down by over 40% during the previous four decades. In 2013, the anticipated response rate to the national Norwegian Monitor Survey was just over 8%. Although a lack of participation may lead to sample skewness, the sample population in this study was within the range of a normal distribution and was considered acceptable (George and Mallery 2010). In this study, the definition of collaboration, learning, and utility was subject to individual interpretation. Measuring occurred based on the instrument’s items, and not a predefined definition of terms. Thus, the participants may have interpreted the meaning differently, which may have influenced their answers and can have resulted in somewhat lower term validity.