Keywords

1 Introduction

Still in 2013, Moreno et al. stated that “the integration of usability engineering methods into software development life cycles is seldom realized in industrial settings” [11]. One reason for this is the “sheer lack of usability specialists in the industry” [5], which results in insufficient knowledge about the work of the end user [8] and in the so-called “developer mindset” [1], overly focused on technological aspects. Another issue relates to the limited suitability of most usability and UX methods for the Agile setting [15], with several authors [2, 4, 7] reporting a particular scarcity of lightweight practices for user involvement in development projects despite the benefits induced by the ability to perform usability and UX work in an agile context [4, 15]. In addition, even companies realising a need to increase the usability of their products may be unable to invest in the resources needed to achieve this [5], and this is particularly true in the case of small enterprises [1, 5].

To facilitate the adoption of user-centred design (UCD) in small Agile companies, we curated the identification of a small set of design techniques; we then planned an action research intervention for presenting them to developers and assessing the impact of these techniques on their working practices. A first iteration has been reported in [3], while a second iteration is reported here. Our results suggest that even such a lightweight approach may support the enactment of a user-centred mindset. However, the impact of the intervention has been limited by the relationship with a dominant customer resistant both to Agile and UCD: we conclude by pointing out the need both for researchers and practitioners to investigate more effective ways to communicate the business benefits that the two approaches may bring.

2 Related Work

The term “user-centred design” denotes a broad set of techniques, methods, procedures and processes that places the user at the centre of an iterative design process [17]. The acknowledged benefits of involving users in systems design [e.g. 1] include improved quality and acceptance of the system, and cost saving [12]. Although promising to support “the execution of software development projects targeting the delivery of useful and usable software” [4], the integration of UCD and Agile development is however not trivial to achieve [e.g. 2] and limited empirical research exists on the topic [4, 7]. One of the ways to enact this integration, particularly in the limited-resource context of small enterprises, is “to use the software developers as a UX work resource by enhancing their qualifications within the field of usability and UX” [14], or in other words to train developers on usability techniques. Advantages include “the potential of easing problems regarding the lack of usability specialists in the industry” [5]; the chance for small companies to lessen “the need to staff usability specialists, which cannot be funded” [5]; a good fit with the Agile feature of team members being able to perform every given work task [13]. This is where we place our contribution.

We also point out, however, that also the customer needs to be supportive of the integration of UCD and Agile, allowing for a suitable design to be researched [2] and for adequate access to users. Scepticism is more frequent in large customers [10] and may result in a lack of customer engagement, which can be a big challenge for development teams [10] especially given the relevance placed on the customer by the Agile philosophy. The solution may require the capability of demonstrating business value, management support, and nurturing a change of mindset and culture in the customer [9]: how to effectively communicate this has however remained largely an open point to date.

3 Action Research Intervention

The activities described here were carried out in “the Company”, a branch of a large Italian IT group providing cyber security and network configuration services to the largest telecommunication operators of the country. The Company had long adopted Agile successfully, and had one main customer, a large telecommunication provider that we will call “the Telco”. Being the Telco a much larger venture, the power relationship between the two parties was naturally asymmetrical, although generally warm. However, the Telco is also a highly structured corporate, whose constrained workflow prevents a full implementation of Agile in the projects followed by the Company, and where some representatives seem to oppose contacts between the Company and final users of their software. While trying to reduce their dependency from the Telco, the Company realised that they were lacking sufficient skills in usability and interface design, and that this could be an issue in proposing their products to fresh customers; therefore, they asked for our help.

3.1 Method

We followed the Cooperative Method Development approach, a “domain specific adaptation of action research” that moves from an ethnographically-inspired understanding of the “existing practice of software development in concrete industrial settings” and aims at improving such practice by cooperating with practitioners [6].

Design techniques presented to developers were chosen to overcome potential communication breakdowns in the integration of UCD and Agile [2], and to reflect surveys on the usability techniques most used in industry [e.g. 14], particularly accounting for their feasibility of integration into an Agile environment and of teaching non-UX professionals. These methods include low-fidelity prototyping, usability testing, personas, expert evaluations, and user task analyses. We remark that this intervention is meant for “supporting developers during ongoing day-to-day product development” [13] and that “we do not discard the need for a usability expert” [8].

3.2 Preliminary Understanding

The first author interviewed developers in June 2016 about their perception of the working environment, their current working practices, and their attitude towards UCD-related themes. The interview study lasted two days and involved 7 people. For what concerns the organisational setting, the Company employs about 20 people, mostly young graduate developers, and exhibits a pretty flat hierarchy. The environment is predominantly technical, yet with a positive and rather curious attitude towards UCD-related themes, to the point that employees explicitly argued in favour of the collaboration with our University in front of the group managers, who tend to adopt a more “command and control” approach instead.

The Company proposed to focus on what we will call the Software (a desktop application used to configure and monitor networking devices for corporate customers of Telco) as a running example during the intervention, for a variety of reasons: it is entirely developed within the Company for Telco; it has evolved over several years as the juxtaposition of different parts, and would now need a refactoring; being one of the main projects of the Company, it is sufficiently well known to all employees.

3.3 Implementation

In August 2016 a series of four workshops, each lasting a whole day, was carried out at the Company site. The agenda of workshops is outlined in Fig. 1 and was grounded on different elements: on a practical level, we accounted for the results of preliminary interviews and for the feedback from our previous iteration of a similar series of workshops [3]; on a more theoretical level, we accounted for the stages of the traditional UCD lifecycle, and the set of focal points to consider in the integration of UCD and Agile development [2], namely the extent of user and customer involvement, the role of documentation, the synchronisation of design and development iterations, and ownership over UX tasks.

Fig. 1.
figure 1

Workshops overview.

Full size image

Three developers (who will be referenced as D1 to D3) were appointed to attend the whole workshop cycle, led by the first author in the role of a facilitator. All of them expressed great interest in user-related themes: D1 was self-taught on UCD techniques, while D2 and D3 were not familiar at all with them.

“One doesn’t even know where to start from, without knowing any basics” (D3)

“More than once [design choices] have been a stab in the dark” (D1)

“If there’s one skill in the Company we are really lacking it is interface design … we try to do what we can” (D2)

Workshops started by motivating more formally the advantages of adopting UCD, that is by presenting well-known reports from industry [e.g. 18] highlighting user involvement as a key factor for project success, and in contrast the lack of it as one of the most common reasons for failure. We then considered the workflow supported by the Software, illustrated in a very technology-centred way in its user manual, and guided participants in re-elaborating it focusing on the perspective of users. In collaboration with the facilitator, participants then outlined the stakeholder network related to the Software, which confirmed how the needs of actual users were generally mediated when reported to the Company, if collected at all. A task analysis was then performed for actors most likely to interact significantly with the Software, and was represented through use-case diagrams. The project manager was chosen as the reference user: information on the characteristics of Software users in this role was retrieved indirectly, i.e. through LinkedIn and narratives of other Company employees, and then expressed through a couple of personas representing different levels of expertise; these in turn inspired scenarios and storyboards.

Once a reference persona was chosen, participants rated the dimensions of usability listed in [12] through poker planning, regarding them as non-functional quality criteria for the Software. Participants then elaborated different low-fidelity prototypes for a specific interface of the Software; however, a later inspection revealed that these alternatives could not support the same workflow articulation as the existing interface. Hence, since the latter was anyway rather complex, participants asked for support in wireframing a more logical re-grouping of its functionalities.

Finally, the different purposes of low- and high-fidelity prototypes and how to communicate them were illustrated, since D1 repeatedly pointed out that Telco would not accept discussing over a “non serious” low-fidelity prototype and that previous attempts at doing this had failed. In addition, a session of user testing was simulated on the ERP system in use at the Company for demonstrative purposes. After the end of the intervention, participants organised a wrap-up session and, a few weeks later, a dissemination seminar for their colleagues.

3.4 Evaluation

In December 2016, an external researcher interviewed participants about what they remembered of proposed techniques after a few months and whether they felt that their approach to design and development had changed. Interviews were loosely transcribed and thematically analysed. Overall, participants positively welcomed our intervention, regarding it as a chance of professional growth. They appreciated having learnt concrete techniques, and remembered them correctly:

“I enjoyed wireframing a lot. It really gave me a different point of view” (D1)

“We should organise the info with wireframes, the poor user will be scared” (D3)

In addition, they expressed appreciation also for the presence of a trainer, reiterating the effectiveness of scaffolding [19]:

“In terms of common sense, this is what every developer should do. Yet having someone explaining you the steps to follow is something different” (D2)

“Now I have a method”(D3)

The training seems in fact to have contributed to enacting a shift from a technology-focused mindset to a more user-centred one:

“Before we used to say – [the user] will have to get over it” “The interface as the means to achieve an objective from the user’s point of view” “The goal is to remove the need for a manual – even for us as developers!” (D3)

“We’ll surely follow this approach rather than – bah, let’s just do something” “I have been assigned to a project where the interface is set in stone [by Telco], BUT [developers and management] all agree that we are going to apply UCD techniques at the first suitable occasion” (D2)

D2 and D3 in fact claimed to have applied proposed techniques as much as possible to the improvement of minor parts of the Software interface that had been assigned to them, and to have used them to support communication with colleagues:

“Prototypes and scenarios can be used internally to understand how to design something […] I proposed some prototypes to my colleagues and this simplified the discussion” “In my opinion personas should be shared by the whole team… to raise awareness among colleagues” (D3)

Participants also commented on the positive attitude shown by the rest of the Company at the end of the dissemination seminar they led:

“We reported to the rest of the Company and the reaction was – let’s hope we will soon have projects where to apply this approach” (D2)

Despite the satisfaction and interest shown, participants did not believe the approach would prove fully applicable in the interaction with their customer due to the strong “developer mindset” [1] of Telco’s representatives, even harder to overcome due to the unbalanced power relationship with the Company:

“Personas cannot be used with Telco: our customer is very much feature-oriented and in a dominant position […] it does not want to see the prototype, it wants to see the product” “Some techniques will be more applicable than others, because it is impossible to access users […] We have no [user] feedback. Clearly we miss it” (D1)

“I guess the customer would be disappointed by storyboards on paper […] it may think we did not put too much effort into such a proposal” (D3)

4 Discussion

In terms of the applicability of the presented approach to other small enterprises, we suggest that, together with the Company’s “culture receptive to UCD” [2], developers’ consolidated familiarity with Agile (including being used to change and flexibility, iteration, and frequent interaction with the customer) may have allowed a deeper appropriation of UX techniques, resulting in a potentially sustained impact over working practices. Furthermore, participants demonstrated an accurate recall of techniques and of their rationale, and reported a spontaneous sharing of their learning with colleagues, applying proposed techniques whenever possible to support interface design and internal discussion. This reflects claims in [16], where Agile and UCD-inspired practices are considered to have a positive impact on mutual understanding and communication; moreover, these factors suggest that even a lightweight intervention such as the one described in this paper may support the enactment of a more user-centred mindset. This can constitute a first step for the organisation towards the awareness of the benefits of integrating UCD, providing elements to decide whether to proceed in developers’ UX training or to hire a specialist designer.

The impact of proposed techniques seems however to have been limited by the Company doing Agile in a non-Agile environment, where this label includes both the culture of the parent group and of the Telco. We argue that the same challenges encountered in this setting [9, 10] can be found when introducing UCD in an environment not accustomed to it. We envision as future work the evaluation of the set of UCD techniques in an Agile company whose customer is also Agile: this would be the most favourable context. In conclusion, we point out to the research and practitioners’ community that there is still a lack of suitable ways to clearly communicate to reluctant customers the potential benefits of Agile and UCD [10].