This section discusses our research findings and implications for researchers and practitioners.
Table 5 Overview of themes in relation to case companies and related work To summarize our findings with related work, Table 5 shows an overview of themes and topics. We also name the cases in which the themes were discussed most prominently.
Based on the discussion of our findings, we arrived at practical principles for collaborative traceability management. We motivate the principles in Sects. 8.1–8.4, where we discuss each of the research questions.
Challenges
We answer RQ1 based on our findings: What are practitioners’ challenges with collaboration in traceability management? We identified three groups challenges that we summarize and discuss in the following:
Collaboration across boundaries
Our findings indicate that collaboration across team and discipline boundaries is challenging in practice. The issue of collaborating in distributed organizations has been confirmed by related studies. Sinha et al. [54] mentioned challenges related to globally distributed requirements management, many of which we found to also hold for traceability management: There is an impact not only because of geographic separation, but also because of different disciplines and backgrounds, used tools, and organizational separation of departments. This separation is partly due to separate groups of traceability stakeholders with different viewpoints having unaligned goals—for instance, between upper management and engineers [23]. Other issues found by related work are organizational problems, e.g., politics or lack of training [23].
Kirova et al. [24] analyzed the cost of traceability and the challenges of tool boundaries and heterogeneous tools. Our findings confirm the need for better support for analysis of trace links and flexibility when it comes to diverse paradigms and tools.
Asuncion et al. [3] addressed this challenge by centering the data exchange of their software traceability solution around a shared database accessible by all tools. However, when different companies have to collaborate this can be infeasible due to legal issues, as we described in Sect. 4.1.2.
Common goals and responsibilities
We observed a lack of common goals and responsibilities, which is strongly related to the lack of benefits of traceability. To justify the effort of traceability management, it is relevant but often challenging to convey the benefit of traceability to involved stakeholders.
We found that the creators of trace links are often not the users of information. It is difficult to motivate stakeholders to invest in trace link quality if the information needs and goals of traceability are unclear. This is what our first principle addresses: (P1) Put stakeholders’ information needs and goals of traceability at the center.
The alignment of viewpoints and goals of traceability stakeholders was also found to be challenging by Kannenberg and Saiedian [23]. In fact, we found that the core of the issue is that the people leveraging trace links are typically not the stakeholders creating the links.
Our findings confirm the problems with traceability mentioned already in 1994 by Gotel and Finkelstein [18]: They found that invisible responsibilities complicate the exchange of information among parties. We identified the issue that responsibilities for traceability are often not clearly defined when organizational boundaries need to be crossed.
Arkley and Riddle [2] also stressed the lack of perceived benefit of traceability to stakeholders, especially to the trace link creators. Another issue was that information in the traceability tool had to be entered multiple times. These experiences support our findings that tool support impacts how traceability is perceived and used.
Panis [37] discussed several challenges related to traceability degradation, the benefits of traceability, and stakeholders’ motivation. He concluded that traceability needs to be “automatically visible for engineers as part of their daily work” to motivate them to improve trace link quality. Similarly, our findings suggest that beyond seeing traceability in their daily work, stakeholders are much more likely to improve trace link quality, when they see the benefits of traceability.
We interpret collaborative traceability as a special case of experience and knowledge management [49]. From this perspective, it is clear that an experience bearer (e.g., a stakeholder with knowledge about a dependency between requirements) is more likely to share knowledge (e.g., by fixing a wrong trace link), if the benefit of doing so exceeds the effort. In this context, Averbakh [5] distinguished lightweight approaches, which aim at reducing effort, from heavyweight approaches, which aim at maximizing benefit. Averbakh recommended minimizing the time required to share knowledge (i.e., maintain traceability) and shift effort away from the experience bearer to others. The latter is also crucial for traceability: Roles that possess valuable knowledge to be captured in trace links (e.g., developers) are often not the ones that have the main benefit. For this reason, we arrive at the following principle: (P2) Balance the effort and benefit of traceability management per role.
Collaborative trace link maintenance
Trace links and trace link quality are often directly affected by changes of connected artifacts. Therefore, trace links and artifacts need to be maintained, which can be challenging if collaboration between stakeholders is insufficient. In this context, we found especially that a lack of change notification and propagation can impede traceability maintenance. Besides trace link quality, it also negatively affects collaboration across organizational, discipline, and tool borders. For this reason, we specify the following principle: (P3) Enable change propagation and notification across boundaries.
Sengupta et al. [52] found similar challenges with the propagation and management of requirement changes in general. Requirements changes typically have an effect on design and coding, which is where traceability could be leveraged as a facilitator. We expect suitable change notification features to support not only trace link maintenance, but the general issue of change propagation.
The topic of trace link quality has been addressed by Rempel and Mäder’s [43] model to assess requirements traceability. Our findings indicate that it is central to analyze quality of trace links. Ongoing research investigates how the quality of manually created and potentially untrustworthy trace links can be improved (cf., e.g., [57]).
Effects of traceability management on collaboration
In the following, we discuss our results for RQ2: How can traceability management support collaboration?
We found that collaborative issues can be mitigated with adequate traceability management. This requires that collaborators have common goals, and see that the perceived effort of traceability is exceeded by the benefit of overcoming such boundaries. The issue of tool boundaries could be facilitated by tool suppliers that move toward common standards such as OSLC.Footnote 7 Whereas tool boundaries and the lack of applicable tool support play a role in practice, they are not the only complicating factors to be blamed. More severe is the issue of invisible benefits of traceability which makes it difficult to leverage traceability in practice. Therefore, we see it as imperative to create direct, tangible advantages for engineers that actively maintain trace links by allowing them, e.g., to more easily identify the impacts of change or to engage other members of their distributed team, and thus ensure that a positive cost–benefit balance is perceived by all stakeholders. Our findings indicate that traceability can be used to enable collaboration and facilitate communication and knowledge management, especially in distributed teams.
Our findings indicate that traceability management can improve collaboration in the following four ways:
Easier communication in distributed environments
Based on an evaluation of a distributed requirements management tool, Sinha et al. [54] identified contextual collaboration as a useful feature. Contextual collaboration allows stakeholders to initiate conversations that include links to requirements and other artifacts. Our findings confirm that it is indeed beneficial to have better support to collaborate in the context of artifacts or trace links, especially in distributed teams and for collaborative traceability management. It is important to note that users must be comfortable with the provided tool solutions. Leveraging existing communication mechanisms known to the user is thus important [54].
Based on our findings, we conclude that in distributed environments, developers would ideally introduce and maintain useful trace links as a by-product [48] of the necessary project documentation (e.g., by allowing references to specific potential trace links in chat rooms). This would reduce effort, while improved communication would increase the benefit of traceability management. Thus, the effort/benefit ratio would be improved, and developers would be more likely to contribute to traceability management.
The explicit documentation of decisions
Ramesh [40] described that traceability plays a crucial role for knowledge management processes. Among other advantages, trace links can support the documentation of critical design decisions or assumptions.
Our study confirms the need for more explicit documentation of decisions and the potential of traceability to support it. Unclear decisions can block developers. By increasing the explicitness of decision that relate to relevant artifacts, the benefit of traceability is increased for developers, who are then more likely to invest in trace link quality. Traceability is thus an important enabler for documentation and knowledge management.
Creating trace links to receive information
As the suggested change notification feature ensures that responsible stakeholders are informed about relevant changes, it could actually improve and simplify collaboration. Helming et al. [21] suggested a model-based change awareness approach, identifying which users to notify based on trace links. This was evaluated in a case study and found to be indeed useful in practice. We can confirm that practitioners are interested in such features. With more agility and moving toward continuous integration and deployment, technical dependencies between teams will increasingly appear at a later stage [51]. Thus, we expect that developers will more likely benefit from receiving information about changes, shifting the effort/benefit relationship to their favor and making it more likely that they will contribute to trace link quality.
Interdisciplinary engineering
Dependencies between different parts of a system can be captured explicitly using trace links, helping users to determine which disciplines must be consulted when certain changes are made. Our study suggests that traceability can positively impact collaboration as well as change management in interdisciplinary contexts. This enabler of traceability was only mentioned by one interviewee, a system software architect from Case 14. This theme was emerging from the interview and not part of our interview guide. A potential explanation for the low number of interviewees reporting on this theme could be that the other interviewees’ focus laid more on one discipline, rather than systems engineering. Future studies could directly address this theme and further examine the potential of traceability for interdisciplinary engineering.
However, we found that a number of related studies confirm our finding. Königs et al. [26] concluded, based on an analysis of current systems engineering practices, that traceability can provide substantial support for interaction and communication in interdisciplinary scenarios. This claim is supported by investigating two tools supporting traceability management in systems engineering contexts. Similarly, Jaber et al. [22] presented the results of a survey focusing on what kinds of trace links different stakeholders are interested in, and what artifacts need to be connected. Through experiments they demonstrated that these links are useful for supporting maintenance tasks and for fostering collaboration between business and technical stakeholders.
Traceability management approaches
In the following, we discuss our findings concerning RQ3: How does collaboration relate to different approaches of traceability management?
We found that there exist (1) requirements-centered traceability management, (2) developer-driven traceability management, as well as (3) mixed approaches, combining the former two. As the names suggest, requirements-centered traceability management is focused on requirements as the main artifacts, whereas developer-driven traceability is mainly conducted and leveraged by developers. Organizations typically introduce requirements-centered traceability to assure the quality or safety of their products in a top-down manner. Developer-driven traceability management originates from development teams.
In requirements-centered cases, collaboration is often achieved in a formal setting that helps regulate the interactions between a typically larger team and a customer who is often very involved in the projects. Much of this collaboration is facilitated through a requirements management tool, and both traceability and collaboration are focused on the requirements. In contrast, developer-driven cases are usually found in an environment where a small team communicates informally and face to face and where collaboration with external stakeholders is less relevant. An issue tracker is often used as the main locus of traceability, e.g., between commits and tickets.
These findings indicate that the concrete way that collaborative traceability management is implemented in an organization depends heavily on the established collaboration approaches and the team size. To realize the potential benefits of traceability management on collaboration as presented in Sect. 5, a traceability strategy in a more formal setting needs to be equally formalized with strong tool support and potential enforcement by the tool. On the other hand, in less formal settings it is easier to establish norms and rules regarding collaborative traceability management in an ad hoc fashion within the smaller team. Enforcement can be based on social pressure or a definition of done.
Our findings cannot confirm Cleland-Huang [7]’s finding that trace links between requirements and test cases in agile projects are used. In the agile cases of our case study, traceability of requirements to test cases was not mentioned. To some extent, we can confirm Mäder et al.’s classification [30]: There exist “regulated” traceability users, sub-contractors, consultants, and enthusiasts. For instance, the regulated type is in line with the requirements-centered approach, motivated by the need to comply with standards and enforcing processes. Our findings do not suggest that the concrete types of inter-organizational dependencies (working as a consultant or sub-contractor) strongly impacted the way traceability is used. However, collaboration across organizational boundaries did play a role and comes with challenges, as we presented in Sect. 4.1.2.
Contextual factors’ influence
In the following, we summarize and discuss RQ4: What characteristics of the development effort influence traceability management and collaboration?
Similar to earlier studies, we found that traceability management can also be leveraged in agile projects. Based on our data, we could not confirm differences between traceability management in agile contexts that depend on scale, complexity, and safety-criticality that were presented in [7]. This can be connected to the fact that the large-scale, complex, and safety-critical cases in our study came from the automotive domain where the transition to agile paradigms is still in progress. We found that traceability management can be used in both agile and V-model-based contexts. However, our findings indicate that it might be desirable to rather rigorously define and follow traceability management strategies. It is difficult to find a good cost–benefit balance if the practices are followed in an inconsistent way and the quality of trace links drops. We capture this point in the following principle: (P4) Strive for a rigorous culture with respect to traceability maintenance.
We identified several cultural, organizational, and process factors that impact how traceability management is conducted. Differences between approaches can be most prominently observed in the development paradigms and the rigor of following them. Our findings suggest that agile and plan-driven cases focus on different artifacts and purposes of traceability: Cases following agile paradigms tend to focus more on development artifacts, whereas plan-driven cases use requirements-centered traceability. It is crucial to analyze the traceability goals and choose and appropriate approach that fits to the purposes. We relate these points to the importance of stakeholders’ information needs and goals (P1), and capture them in the following sub-principles: (P1a) Choose a lightweight, developer-centric approach for goals related to tracking change to code. (P1b) Choose a formal, requirements-centric approach for goals related to regulation (e.g., safety or legal requirements).
Moreover, the trace link quality and maintenance appear to relate to the culture and rigor: Lenient cases and less well-defined paradigms reported difficulties with trace link quality and maintenance. In case trace links are not established in a systematic manner, stakeholders struggled with identifying the benefit and use of traceability, and thus with finding motivation to create trace links.
Our findings are in line with Rempel et al.’s finding that practitioners need to specify traceability goals, and find ways to implement and assess them [44]. In practice, this is often not the case, which results in inconsistencies between the goals of the development paradigm, traceability goals, and the actual trace links [44]. As a consequence, trace link quality is often too low to be suitable for practitioners.
We found successful traceability both in plan-driven and agile cases. However, only two of the cases followed agile paradigms and had a rather rigorous culture. During the analysis, we found that culture and rigor play a critical role, in addition to the development paradigm. Had we found these emerging aspects earlier, we could have aimed to include more agile/rigorous samples. Future research will have to scrutinize whether this finding can be generalized beyond our sample.