Main Architectural Goals of Practitioners
As a prelude to asking practitioners about AC, they were asked about the core architectural challenges they face in their daily work. The most prevalent architectural goals mentioned by participants were:
Migration of software from legacy systems into services and into new technologies without adversely affecting customers. 79% of our participants mentioned this as a goal (P2, P3, P4, P5, P7, P9, P10, P11, P13, P14, P15, P16, P17, P18, P19). For example P14 stated ‘every four or five years technology changes, new devices come on and we need to be in a position with our architecture that we can move with that without huge cost and huge impact’.
Scaling software to larger user bases, better performance, and big data without adversely affecting customers. 42% of our participants mentioned it as a goal (P5, P13, P12, P15, P16, P17, P18, P19). For example, P16 noted the complexity of the user-scaling issue: ‘when there is a threshold of millions of users you may (even) need to redesign problems that were not possible to foresee before….the (whole) system can be replaced’
Releasing different products or versions from the same code base. 42% of our participants mentioned it as a goal (P1, P12, P13, P14, P15, P16, P18, P19). This is best illustrated by a quote from P14, with respect to customer customization of their system: ‘Every time we want to upgrade someone from one version to another major or even minor version in some cases we are dealing with huge conversion problems. We are dealing with the customization customers have in place... so that when we push out the next version we break all (of them).’
Maintenance of Software: The complexity and coupling within their current architecture was often perceived as holding them back. 32% of our participants mentioned it as an issue (P5, P10, P12, P13, P14, P19). One participant referred to his ‘existing code base that is more or less a blob’ and several others referred to the implications of such complex coupling: for example, P5 and P13, noted that even senior developers had difficulty in understanding some aspects of the system and that they were ‘afraid to touch the code because we change something here and break something over there’.
AC: Current Practises
The practises reported during the interviews to support AC can be categorized into two groups: Communication Practises that communicate the architecture across the team and System Practises that aim to avoid, minimize, or detect architectural inconsistencies in the system’s implementation through technical means.
AC: Communication Practises
Participants’ perceptions of the level of architectural knowledge currently in their teams varied: five replied that the knowledge was not well-distributed at all, and three mentioned that senior software engineers are more knowledgeable about the architecture than more junior ones. In the forward direction (from architecture to implementation) this was sometimes attributed to the separation of architectural and development teams. But P1 also noted a considerable delay in development-team insights being relayed to and considered by the architectural team, a significant frustration for the development teams involved.
In terms of the practises used to make development teams more aware of the architecture, the following options were mentioned by more than one participant:
The usage of wikis, skype, forums and sometimes training to share architecture knowledge;
Conducting Scrum meetings. Usually they focussed on shared code, but hardly any diagrams;
Having an architecture steering committee within the company that is aware of what each team is doing, with architects from each development team on that architecture steering committee.
Providing heavy-weight architectural documentation to the development teams.
It should be noted that, of the communication channels between the architects and developers, the first two options mentioned are quite informal, consisting of wikis, forums and scrum meetings. In addition the communication channel is often transient (scrum meeting, skype) or persistent, but de-localized from the developers’ work context (wikis, forums, documentation).
AC: System Practises
Despite the barriers to adoption, practitioners did mention that they attempted to minimize architectural inconsistencies by:
Tools for ensuring rules are fulfilled in code. P1, P3, P4, P5, P9, P10 and P11 mentioned that they use SonarQube (2014) which allows the definition of non-dependencies between source code structures like packages. All of these participants proactively noted though that Sonar did not support architectural recovery and did not provide a holistic representation of the architecture for consistency.
Using naming conventions (P2, P10, P13, P17). They mentioned that such conventions reduce inconsistencies, as developers can understand what a package or module does, determine their patterns of usage, and determine if they are stateless or not. For example, P10 said, “It should be possible to tell what role a package plays just from its name”
Code reviews (P7, P11, P13, P16, P18). All apart from P11 mentioned that members of the team manually review the code of other team members for, among other things, AC. P11 mentioned that they used a software tool called Crucible (Crucible 2016) to facilitate the review.
Built in scripting utilities for searching and exploring the code base to determine how individual parts call each other (P13, P14) or to ensure modularization. P13 said, “it’s not a big picture thing. It’s really at a lower level".
Declaring the dependencies of each module in the code. P12 said, “we declare what the dependencies of our module is. So if this module requires services from some others we can specify what that interface is and then there is no way someone can call something even though it is public from another module if it is not part of the interface”
Usage of model driven engineering code generation tools that follow the design (P13). P13 said, “we would forward generate and then we would make changes subsequent to that. There would be a whole bunch of annotated tags in the code that you could reverse engineer back to Rational Rose’s again (so you can say) hold on what’s that happening there”.
Many of these system practises focus on informal approaches, manual approaches or on specific dependencies, rather than having a holistic architectural focus over the whole system. In the next section we look at the barriers practitioners perceive in arguing for a more holistic, formal AC approach to become part of their organizational culture.
Barriers for Adopting More Explicit AC Approaches
Several difficulties in adopting more formal, explicit AC approaches were noted by the participants:
Difficulty in quantifying architectural inconsistency effects, and thus justifying the allocation of resources to fix them, to senior management. P15 tried to estimate the costs:“let’s say 20% of development time is being spent on things that really they shouldn’t have to do at all if you could pull back and redesign the platform, but because you can’t do it you constantly have got to compensate...”. Likewise, P5 mentioned that these architectural problems seem to have tangible costs such as increased time to market, demoralization of staff and a lesser ability to sell systems to potential customers.
But both were very clear that these were qualitative assertions. Likewise, P13 pointed to the same difficulty: “I personally cannot estimate the cost… there is a cost, we have been trying to do this for a long time and we’ve had attempts before. It’s not that easy”. P14 identified a substantial time-lag before any approximation could be made “the value of good architecture is very hard to see... it is not something that you can build it and put it out and six months later say there’s a return. It’s going to be 10 years later or it is going to be 20 years later.” However, P1 did acknowledge that, while it was difficult to estimate the costs, “these issues are visible to senior management” as their code is getting more expensive (in time and money) to maintain.
Architectural inconsistencies are not obvious to customers and this is seen as diminishing their importance. For example, P13 said “…it should be easier for us to migrate into it (new technology with a simplified architecture) which is a perception I guess. But they (the customers) won’t see it straight off. When a customer uses a screen, they don’t know what the architecture is behind it.” In support, P15 said, “The customer cares as long as it’s quick and it gives them what they want. And so then how do you justify in the short-term changes that don’t impact the customer... As long as that’s an issue and not easy to do, it doesn’t happen in reality. Because people are under pressure to get stuff out of the door they don’t come back to update that"
However, P1, P14, P17 and P19 comments implied that customers are becoming aware of the issues: P14 said, “…you have the suffering and pain of not being able to give them an easy way to upgrade”. Likewise P17 said, “Customers feel architectural challenges as bugs and slower turn around on features”.
Reluctance in fixing architectural inconsistencies. Feeding on from the previous point, P13 stated that his organization would be reluctant to fix inconsistencies unless it impacted on the customers: “It probably isn’t ideal but it isn’t going to affect the bottom line of our customers.” But that wasn’t the only reason cited for inaction. For example P14, who was from the same organization as P13, said: “we are reluctant if code was written and it works: we don’t want for somebody to come in and rewrite it into a better architecture”, suggesting that the problem is in trusting the quality of the change and its effects on the code-base.
Likewise P15 said that inconsistencies cannot be fixed because once an inconsistency is introduced, there will be other dependencies built around it, and that possible ripple effects will occur: “the problem is when you build these dependencies it is like a rip in an old jumper you pull a little knit that you need and then you see the whole application comes with you because this depends on this and then there’s a reference to one of these in here you will never use it, and then the application won’t compile.”
The effort required to map the system to the architecture: With respect to RM approaches specifically, practitioners noted the difficulty of mapping code elements to architectural concepts as a precursor step. When the systems code structure or lexicons are not aligned to the planned architecture the mapping is viewed as a more time consuming process (P4, P8, P9, P16). In these instances, participants noted that the effort they would have to expend in architecture recovery is excessive. In addition, when there is a large code base, it is difficult to generate well-informed assumptions on how the code is structured/named with respect to that architecture. However, even when the code is strictly aligned to a planned architecture (e.g., packages are aligned with architectural components) the architect must first generate the planned architecture. Referring to this issue, P9 said, “You still have to create the categories for grouping – i.e., the components”.
Software Development Situations where AC is Considered Useful
In this section, we present different situations, identified by practitioners, where they envision AC to be a useful tool in supporting software development. These situations are:
Increasing Architectural Knowledge Awareness
Fifteen participants mentioned that using an AC tool would help increasee the architectural awareness of the team and serve to train new project members or junior developers. These members could be new project leads, managers, or developers. For example, P5 noted that AC information would be: “very useful for a new development manager or team lead… before taking over responsibility for an existing project.” Likewise, P3 noted that it could “train more junior developers to be more aware of the architecture design. Currently they can concentrate so much on the details of the code that they lose the higher-level context.” P1, P13 and P19 made the point more generally, for the whole team. P1 said, “An AC tool would help to share the architectural view with everyone in development” and P13 concurred: "… it just encourages the discussion".
Referring specifically to the JITTAC tool demonstrated to participants before their interview, P14 liked the live information provided by the tool, comparing it favourably to documentation. He noted that “it’s a live picture of your architecture rather than a page somewhere that may not have been updated in six months or something like that". He also said that it can help anyone, not just junior developers, because sometimes (more senior) developers work in different projects, or members move from one project to another.
Stopping Inconsistency Introduction
Closely related to architectural awareness, all participants noted that their architecture was constantly under pressure. For example, P13 noted problems caused by the scale of the software system, “Because the code base is large, if people want to get something they just make a direct call and make direct calls to database tables when they should really be going through interfaces”. Quotes from P14 and P15 focus more on deadline pressures: “a lot of the time a developer is working on something they are under pressure. There is a deadline they have got to get this work done and if I make that call it will work and I will get my solution out there.”… “probably the last thing on their mind is architectural consistency. They are just looking at just I’ve got get this out the door…”.
Sometimes though, the issue was more to do with lack of enforcement in the company. For example P14 said that they do not enforce internal architecture of modules, “Within those modules there is also a structure which is like soap modules or packages or so on. I think… to keep those clean and have an internal structure because right now within that module we are not making any conditions about what can or cannot call anything else. AC information was seen as a means of prompting renewed interest in enforcement and prompting developers to take the architecturally consistent, rather than the expedient, path.
Auditing In-House Code
All participants stated that evaluating architecture consistency through AC approaches could be usefully employed in auditing the quality of code. Several situations were identified where AC could be used for auditing the quality of in-house systems:
To periodically determine the quality of an in-house project within the team.
For internal contract acceptance. For example P1’s organization has an analysis/design team and a development team. He noted that the analysis team could use the approach to validate what is actually delivered by the development team.
To check the quality of releases before delivery to a client. For example, P3 said that it could be used as a quality mark, certifying a clean architecture design and consistency to that design.
Auditing for Outsourcing
Eleven out of the 19 participants mentioned the auditing benefit with respect to ensuring the quality of outsourced projects. For example, they noted the following outsourcing disadvantages that AC evaluations may (at least partially) address:
The outsourcer can lack deep experience and culture around architecture design and coding (P9).
P7 noted that contractors are less incentivized to produce the quality of code required to reduce long-term costs.
The outsourcing teams can be less aware of the relevant architecture knowledge. For example, P2 said, “In each (internal) team, there’s a good spread of knowledge of the architecture design. This usually doesn’t hold true for outsourced projects”
In this context the outsourcing client could use the approach to monitor the progress and quality of an outsourced project, and the quality of the final delivery (P5, P10). P12 said, “if you have got your contract in and he has made changes and (they) fail the build... the contractors would be getting feedback immediately as well if they have broken the build in that fashion.”. P2 and P9 viewed an AC approach as one element in a suite of auditing tools including Sonar (SonarQube 2014), testing reports, copy-paste-detection tools, and simple build analysers.
The advantages also apply for the outsource provider. For example, P3 whose "company, from time to time, takes on the maintenance of a client company’s code base" saw “a value” in AC tooling "in helping to judge the quality of the code (especially when an as-designed architecture has been given) or to produce an initial design when it isn’t available.”
Surprisingly, P14 noted the ability of AC tooling to reduce the amount of auditing required by customers of his company’s outsourcing. He said "if they understand that the architecture says that the change I make here has a limited impact and they can see then we don’t need to re-certify everything, then that would reduce the effort on their part. If I test this piece here then I don’t need to test everything else over there because it’s not dependent upon it or whatever.”
Evolving the Architecture
In a similar vein, 15 of the participants mentioned that once an implemented architecture is made consistent, there may be a need to support the process of moving to a new planned architecture. P14 said that AC tools could help in architecture evolution when components have to be replaced or when adding new functionality, as they help in assessing how components depended on others: “a way we can address... what is the impact if we add a new API here or we change the nature of an API. What is the impact of that on the other modules." Likewise P16 said, “If you want to replace a component or a system you have to know which are the interfaces or endpoints that component or subsystem it uses.”
Specifically, with respect to the JITTAC approach P15 said that real-time architecture feedback is essential for architectural evolution. He said, “I think on top of that it enables architectural evolution,…, You can work away and you know that if you break anything the environment is going to let you know and that is key... and that supports I think architectural evolution during development. Without that is very difficult to achieve.”
Features Required for AC Tool Adoption
We asked practitioners, “Would an AC approach be a “must have” or a “like to have”?” 16 out of the 19 participants stated that it was a ‘like-to-have’. We then asked these 16 participants, “What are the features that are needed to make it a must have?”. For the participants that answered “must have”, we asked them, “What are the features that would increase AC adoption?”.
In the following text, we present the features that practitioners stated were needed, in response to these 2 questions. The list is ordered based on the number of practitioners who requested the feature, as a proxy for the feature’s importance. It should be noted though that 4.5.3, 4.5.4 and 4.5.6 received the same importance according to this proxy measure.
Expanding Beyond Source Code and to Deal with System Complexities
All participants agreed that getting dependencies from statically analysing source code only, was an issue for them. P6 said, “The tool would have to understand all dependencies between code components”. Adding to this, P9 said, “It’s a real challenge today to understand the relationships between code modules given the number of different ways there are of introducing relationships. It’s a real challenge to get an overview”. The participants expressed the kinds of dependencies that AC tools would need to support:
Dependency Injection in popular frameworks such as Spring, EJB, Hibernate or Struts. (P4, P5, P6, P9, P10, P11, P15, P16, P18, P19) For example, P16 said, “some concepts cannot be mapped to packages or classes: There are for instance like other kinds of architectural frameworks that do not follow that structure.”
Sometimes single systems are developed in several languages and AC tools should be able to cope with that (P1, P2, P4, P5, P13, P14, P19). P14 said, “we would use Progress and we use C#. There may be other languages that we want to bring in or there may be ones even on top of Java like Scala or some of those so it would need to I think probably have a broad and possibly even need to interact between languages.” The ability to interpret bytecode to be able to analyse languages built on top of Java Virtual Machine such as Scala was mentioned by several participants (P2, P9, P11, P14).
Web Services and RESTfull APIs (P2, P3, P6, P7, P8, P15, P16, P18, P19). Support for service oriented architecture was desired. P16 said, “… only takes into account packages and classes. For instance, for web service architectures… that I am using this service or this or this.”
Annotations (P9, P12, P13, P14, P16, P18). P12 noted their importance when he stated that “They radically alter the behaviour of your software. They are not just documentations they enact code changes at runtime and compile time”
Metadata represented in XML files (P4, P9, P12, P14, P19). For example, P14 said, “Our API covers that but this is represented in an XML file... it’s exposed for use by a client but it doesn’t take the form of parameters to a function.”
Dealing with Databases access from source code e.g., SQL, RDBMS (P12, P19)
Just-In-Time/Real Time Inconsistency Awareness
Twelve of the participants liked the idea of real-time alerts, although this may have been biased by the (JITTAC) tool chosen for the demonstration. Expressing a desire for a more real-time approach P13 said, “The sooner you get that, the better because there is no point in discovering that after he’s done a lot of work and then getting that to be reworked. So the earlier in the development process you can detect an issue then the better. It costs less.”
Many of the participants (P3, P5, P9, P15, P16, P17, P18, P19) liked that this facility would allow them to develop the code and architecture in parallel. For example, P15 said: “I like it to develop architectural in parallel with code because... it takes an element of communication out. It removes the need for the architect to come down and tell people that he has made a change because they see it straight away. That’s great stuff.”
Likewise in a more bottom-up perspective, P14 said that the facility would be good for integrating design and coding at the same time. He said, “a lot of the time I do design in the code and I will try something out and I will see what it looks like…. So I would use it in design because then I can see what the impact of some change I am about to make is on something else.”
Support for Fixing Inconsistencies
Some participants stated that remediation support has to be provided when inconsistencies are identified (P2, P3, P4, P5, P10, P12, P17). The participants described this support in terms of the following:
Managing the process of fixing inconsistencies. The workflow (of fixing inconsistencies) within teams should be explicitly reflected. An example is to allow senior-developers / architects to prioritize certain inconsistencies and mark others as being ok, as an initial step (P2, P3, P4, P5, P10).
Providing some suggestions for fixing inconsistencies e.g., actions or solutions to fixes. (P3, P4, P12). For example, P12 said: “Just say I had a cast in my programme that didn’t use generics, I would typically get a highlight in eclipse saying that this is an unsafe cast that you are making. It would not be a compiler error, it would be a warning. But when you click the little icon on the side of eclipse it says you can change it to use generics or you can add an annotation to suppress the warning. So that is useful to have an action to solve a problem. Similarly it is useful to have such features to solve architectural violations.” P3, P8, and P12 suggested the usage of patterns and anti-patterns to be incorporated in tools to encourage and give developers guidelines on fixing inconsistencies and restructuring the code.
Provide estimations of the costs and efforts for fixing inconsistencies. (P2)
P17 delved more specifically into the utility of ACs for refactoring. He noted that the approach should have several hierarchical architectural models in this context: “refactoring needs to be done between a number of models. You can play with the architecture that you define in <name-of-system> to see what is calling what on a big level. From that you can then divide up your modules, for you, think these are two completely different things right? Redesign in <name-of-system> where component goes and see how you have done."
AC and System Evolution
The participants noted that architectural evolution should be supported by some estimation of the work, effort, risks, time and resultant quality required to move a system from one architecture to another (P1, P2, P3, P4, P5, P9, P10). They stated that it would be good if AC tools were equipped to provide this estimation.
In terms of resultant quality, P1 said, “It would be good if the tool showed some metric to monitor improvements, e.g., moved from a 4 to a 5 (or at least the system didn’t get any worse)”.
In terms of organizing the change and estimating effort/time, P2, P3 and P9 suggested a listing of the tasks required for this move, and help in prioritizing them.
In terms of the risks, P5 suggested that the tooling should show the implications of certain choices. For example, it could show the implications of splitting a component, and then help in tracking the changes that need to be made in the code (P10). Also, the tooling should indicate which areas of the architecture are poor, explain why and indicate how to improve them (P8).
Provide an audit of architectural changes to maintain an architecture timeline that can be browsed by architects for further effort/risk estimation (P5, P10). This could be applied to new developments and to maintenance
Ensure that there is an agreement among architects for a change to e.g., each change must be signed off by two architects (P5).
In terms of easing the effort of evolution, as suggested in section 4.5.3, the tooling should support code refactoring (or change) to achieve consistency with an improved target architecture (Schmit et al. 2011). For example, P2, P4 and P5 suggested teasing out components that are hidden in the code, but which should be isolated to improve the design. P8 suggested the alternative of the tooling offering pattern-based advice, when moving towards target architectures.
Incorporating AC Into Collaborative and Versioning Repositories
Through repositories P1, P5, P9, P10, P12, P13, P16, P18 and P19 noted they could get a set of advantages: different versions of the architectural model would be available to track and could be available to all for collaboration; the user names of those who made changes and performed commits to code could be identified and the developers would not have to work with the whole code-base. P12 stated, “If they are using SVN or GIT or CVS you have a blame functionality that will tell you the line of code checked in by the developers. At least you get the username that way.” P16 also said, “I would do just the change and then we should have also versions of the architecture and before committing to this central branch... the architecture is reviewed and then this update against the model in the main branch can be accepted or rejected”.
Incorporation into the Build Process and Continuous Integration
P15 noted, “(you should) Get it into the continuous integration environment so that you don’t have some poor unfortunate who is sitting there trying to make sure that everybody does the right thing. If they do something stupid it breaks the build...” Likewise P12 said “you expect the feedback to come through your build processor ANT, MAVEN or whatever you decided to use...I would use something that was part of my build process that would run every time... as part of a continuous integration.” In total six participants suggested this requirement (P8, P9, P11, P12, P15, P16, P18, P19).
Defining Richer, Alternative, Architectural Models
Practitioners suggested that it was important to be able to define richer architectural descriptions, constraints or rules. Likewise, support for defining architectural styles (e.g., P10, P19) was seen as an important requirement. But probably the biggest modelling enhancement that the participants desired was to be able to define models around other quality attributes (P10, P13, P16, P18). For example P16 said, “For me the architecture is more defined with the quality attributes than with the structural features, e.g., availability, response time.” P13 added: “So I mean there are other ways which you can evolve your architecture. You can change technologies, you can do things differently to have a higher performance.”
The Incorporation of AC into Organization Culture and Processes
Several participants stated that AC was simply not part of their software development culture: “Adopting an architecture consistency tool is like a habit, you either form a habit or not” (P15). He continued, “I think the important thing would be to get it in early. To get people get used to working with it so that they don’t break the rules from an early stage.” This lack of AC culture means that often organizations don’t have a specific person who would drive this adoption: P13 said, “the setup in the organisation here because somebody doesn’t have overall responsibility for the whole thing... And we probably should have had someone with overall responsibility for dealing with issues like that to help drive that type of stuff.”
AC as Free Open Source Tools
A factor that most participants mentioned is that a tool is easier to adopt if it is non-commercial. For example, P12 said, “you do not need to convince senior managers for having free tools”.