Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

It has long been argued that interactive systems design must consider criteria other than objective and system-oriented ones (e.g., reliability, correctness, effectiveness, efficiency). Over the past decades, we have experienced the introduction of different methodologies seeking to provide designers with means in order to account for more human-oriented criteria, such as human values, user experience (UX) [18], and worth [5], in design projects. Friedman and colleagues have identified several values (e.g., human welfare, privacy, user autonomy, freedom from bias) that should be considered in the design of technology [12–14]. The authors propose Value-Sensitive Design (VSD), a theoretically grounded approach that accounts for human values in a principled and comprehensive manner throughout the design process [13, 14]. Numerous works have been investigating techniques, methods, and frameworks for understanding, designing, and evaluating UX. In another example, Cockton introduces worth [5] and proposes Worth-Centered Design (WCD) for designing worth [5]. The work presented in this paper is related to this last aforementioned methodology WCD.

WCD provides designers with six meta-principles, five “D”s, a framework, and a set of tools, techniques, and methods for designing interactive systems that are worth purchasing, learning, using, and being recommended. If WCD appears, from a theoretical point of view, well suited for the design of today’s and tomorrow’s interactive systems, the design methodology has not received much attention. Only a few researchers, other than Cockton and colleagues, have investigated WCD in design projects. For example, Otero and José applied WCD to the development of digital artifacts for teachers of a secondary school and for children with cognitive and emotional impairments [16]. Vu utilized the WCD framework in the design of an information system, i.e., a web application dedicated to a golf union employees and golf clubs managers [19].

In our opinion, the main reason to this lack of a strong enthusiasm regarding WCD is that the related literature is not very intensive. Moreover, it is spread up over years and most papers address WCD only partially: there is a lack of a sharp compilation on the topic.

This paper presents a complete operationalization of WCD through the design of Cocoon, a context-aware and mobile application. The design project lasted over 3 years and investigated several design tools and techniques, both general and specific to WCD. The remainder is organized as follows. Next section presents a state-of-the-art on WCD. Section 3 relates the worth-centered design of Cocoon, design phase by phase. Finally, Sect. 4 proposes a discussion and directions for future research.

2 Worth-Centered Design in a Nutshell

This section compiles the state-of-the-art on WCD with one subsection per key point.

2.1 Concept of Worth

Worth is a motivator. According to Cockton, “Designing worth means designing things that will motivate people to buy, learn, use or recommend an interactive product, and ideally, most of all these” [5].

It is important to note that “worth” was initially introduced as “value” [3]. However, because worth, as a predicative adjective (as in the usage “because you are worth it”) better conveys the intended meaning of worth in WCD and, in order to avoid confusion with Value Sensitive Design (VSD) [13, 14] for instance, “worth” has replaced “value”.

It is also important to note that worth can be of many forms. According to [3], it can be political, personal, organizational, experiential, and spiritual.

2.2 Six Meta-principles

Cockton proposes the following meta-principles for guiding designers [9]: (1) inclusiveness for taking all stakeholders into account; (2) receptiveness for openness to all ideas; (3) expressivity for a good communication; (4) Credibility of the design to ensure achievement of worth; (5) committedness of the design team for ensuring achievement of worth; and (6) improvability for an appropriate assessment of worth and a good understanding of possible problems.

2.3 Five “D”s

Cockton proposes four D’s for worth achievement assessment: Donation, Delivery, Degrading, and Destruction [5]. According to [5] evaluation leads to Donation when the design offers more than the intended worth. The author highlights Apple products as designs illustrating Donation [3]. Evaluation leads to Delivery, Degrading, and Destruction when the design delivers respectively as much, less, and none of the intended worth.

A fifth D exists within the WCD framework: Denial. In [5], the author points out three causes that can lead to Denial: when causal analysis denies the viability of the worth proposition, the possibility of adequate risk management via appropriate research field, and/or the possibility of any successful design fix with available technology.

2.4 The WCD Framework

The WCD framework [4, 5] structures the design process around four phases: (1) study of needs, wants, and unfelt needs, (2) design, (3) evaluation that may lead to (4) iteration. Figure 1 shows the WCD framework.

Fig. 1.
figure 1

The WCD framework (resulting from the analysis of related works [4, 5, 9])

Full size image

Study of Needs, Wants, and Unfelt Needs

The study of needs, wants, and unfelt needs phase (we will refer to this phase as “study of needs phase” in the remainder of the paper) aims at understanding worth of the system under study. Interviews, cultural probes, competitive analysis, prototyping, envisionment, and performance are examples of techniques that are appropriate for studying worth. In the study of needs phase, involvement of users and other relevant stakeholders is highly recommended. Indeed, according to Cockton, “whenever possible, worth should be expressed using words and images of users, sponsors, and other stakeholders.”

During this initial phase of WCD, relevant meta-principles include inclusiveness in order to involve any relevant stakeholder, receptiveness in order to collect all their ideas and, finally, expressivity in order to convey worth in a way that can be understood by the entire design team.

It is important to note that, in order to enhance expressivity, worth enrichment (i.e., association of direct quotes from interviews, photographs, or video material with worth elements) [6] is possible.

Design

The design phase aims at designing and implementing the system. During this phase, applicable techniques and tools include the ones generally used in interactive systems design, such as mocking-up and prototyping, but also WCD-specific techniques. Specifically, in order to challenge credibility of the design to ensure worth achievement, Worth Delivery Scenarios (WoDS) [5, 7] authoring, worth mapping [6–8, 10, 11], or adapted impact matrices [5] construction has to be carried out.

Briefly, WoDS are stories that clearly demonstrate achievement of worth through design. Stories can be written, storyboarded, performed live, or produced as a video and must have a “happy ending”.

Worth Maps (WMs), resulting from worth mapping, revisit Hierarchical Value Maps (HVMs) [17] in several points for supporting interactive systems design [10]. HVMs associate separately elicited Means-End-Chains (MEC) [15] and are used in Marketing to study customers’ motivations for purchase. In interactive systems design, WMs are visual representations that connect (vertically) system-oriented attributes (e.g., features and qualities) to user-oriented ones (e.g., emotions, feelings), thus shifting from “designing as crafting to designing as connecting” [10].

Finally, the third tool for challenging the system’s credibility in design phase, the WCD version of impact matrices, associates design features with worth achievement.

In addition to credibility, committedness of the design team to produce a concrete product that guarantees worth achievement is the second relevant meta-principle for the design phase.

Evaluation

Evaluation is of a major importance in WCD and should lead to worth assessed in terms of Donation, Delivery, Degrading, and Destruction.

Evaluation should start at an early stage and can employ different methods and techniques. Worth operationalization, i.e., translation of statements about worth into (measurable) criteria, should be part of the evaluation phase activities in WCD. According to [5], the relative role of usability-related attributes has limited relevance when real-world outcomes are achieved. Therefore, field evaluation should be part of the worth evaluation strategy. However, it is worth noting that poor usability may have a negative impact on worth [3]. Therefore, usability testing should also be part of the worth evaluation strategy.

In the evaluation phase, improvability is the key meta-principle for an appropriate assessment of worth and a good understandability of problems.

Iteration

In WCD, if required, iteration can be partial or total depending on worth achievement. Indeed, no iteration is necessary in the Donation case, when expectations are exceeded. However, in order to reach perfection, iteration can be considered in the Delivery case. In the Degrading and Destruction cases, iteration is necessary in order to overcome defects that negatively impact the system’s worth.

Iteration starting point depends on identified problems. For instance, fixes in design are not enough when problems lie in misconceptions about worth and/or in case of Denial. In such cases, iteration from the initial phase, the study of needs phase, is necessary for a better understanding of worth.

3 Worth-Centered Design in Practice

This section reports the operationalization of WCD on a case study, design phase by phase.

3.1 Case Study

Prior to engaging to WCD operationalization, we proceeded to opportunity identification through a review of several existing interactive systems. This lead to the vision of Cocoon, a mobile application that automatically provides the user with different types of information in context. Information is of two types: personal vs. impersonal. Personal information is related to the user’s contacts (e.g., location, a church where a contact’s marriage was celebrated). All non-personal information is considered impersonal (e.g., related to historical buildings and bars; news, music recommendations). Cocoon also provides users with the opportunity to exchange posts, multimedia messages which delivery conditions can be specified according to date and location. Also, Cocoon allows the user to control different parameters: types of received and shared information, numbers of personal and impersonal information notifications on a daily basis, and system activation/deactivation.

It is important to note that our design project did not start with a so well-envisioned system. Rather, outcomes from the opportunity identification phase supported the creation of storyboards illustrating different features for a large design space exploration in terms of target user groups (senior, adult, and young users), platform and interaction paradigms (large display, small display, projector, tactile, speech), and environment (different mobile, sedentary, and social settings). Any criteria suitable to allow users experiencing more than good usability guided the creation of storyboards.

The final version of Cocoon (Fig. 2) was implemented as an Android widget. The Cocoon widget clearly separates personal and impersonal information by two different icons, and gives access to the other features of the system. Information presentation pages look different and offer different possibilities depending on information type (for instance, downloading an item attached to post, accessing to the website of a news provider).

Fig. 2.
figure 2

Cocoon on the HTC Desire S: on the left, the Cocoon widget on the Smartphone home screen; in the middle, a post; on the right, a page of information related to news.

Full size image

3.2 Study of Needs, Wants, and Unfelt Needs

Study of needs is the beginning of our experience with WCD. At this stage, two major questions raised: How to start? And, more importantly, how to reach elements other than the ones related to features and User Interface (UI)? We chose to start with interviews because this technique has been proven efficient for information gathering in general but also in a worth-centered design development context [10]. However, in order to surface elements beyond traditional criteria considered in Human-Computer Interaction (HCI), we carried out a state-of-the art on the laddering technique, used for eliciting HVMs elements [17], for inspiration. Therefore, the study of needs focused not only on the “what” (features) and slightly the “how” (UI and interaction) but also on the “why” (motivations). During interviews, the researcher opportunistically came up with “why” questions on the basis of participants’ statements. In addition, one closing question was asked to participants: “What is the worth of Cocoon from your point of you?”

We conducted 19 semi-structured interviews. Considering our large design space, a wide range of possible users participated to the study: people ranging from 21 to 63 years old (mean: 30), from different social classes and with different professions. Interviews lasted about one hour and were supported by storyboards illustrating the identified features. During interviews, storyboards were first presented to participants. Then, they were questioned in order to gather their comments and feedback regarding Cocoon. Interviews were recorded and transcribed.

Data analysis was carried out by two researchers (the researcher who conducted interviews and another one) and followed 3 steps: (1) extraction of statements related to both positive and negative sides of the system, (2) translation of the extracted statements into well-defined worth elements and association of relevant verbatims with them (worth enrichment), and (3) classification of worth elements into ‘universes’ of worth. We relied on existing value lists and outcomes from evaluation of other existing systems (worth spread opportunities inspection [7]) during worth analysis.

Outcomes About Cocoon

  • Cocoon represents a worthwhile system with regard to different aspects

Data analysis revealed that Cocoon’s positive sides are related to the following four universes of worth: ‘Discovery and Diversity’, ‘Emotions and Feelings’, ‘Exchange and Communication’, and ‘Adaptation and Presentation of information’. Indeed, during interviews, participants mentioned the system’s potential to automatically provide them with diverse types of information (according to context), including information related to relatives. Thanks to this core feature, Cocoon was perceived as a system that allows gaining knowledge in different domains and causes emotions such as pleasure, amusement, surprise, etc.

Thanks to posts, Cocoon was also perceived as a system that encourages to share, and contributes to maintaining social ties as well as knowledge sharing.

Interviews also highlighted Cocoon as innovative in terms of information presentation (for instance, using a pico-projector).

  • Control is a must

If information push was perceived as a worthwhile feature in Cocoon, participants repeatedly expressed their concerns about privacy with regard to personal information sharing as well as about information overload and redundancy with regard to the push mode. Interviews then highlighted the need for allowing control over different aspects in the system (types and numbers of received and shared information, contextual data information collection).

Based on the outcomes from interviews, the functional core of Cocoon was limited to features that were the most voted in by participants. Consequently, our design space was reduced to: adult and young users in terms of target users groups and small display, tactile interaction, mostly mobile settings respectively in terms of platform, interaction paradigm, and environment.

Lessons Learned About WCD

  • #1. Worth is multidimensional

Our study confirms the multidimensional property of worth. Indeed, users stated opinions in relation to different elements, both system-oriented and human-oriented. System-oriented elements included features (e.g. contextual information push) but also qualities of features (e.g., efficient in use). User-oriented elements included consequences from usage (e.g., discovery of new places and new stories) but also worthwhile outcomes (e.g., maintain of ties). In the specific case of Cocoon, interviews revealed that associations of the aforementioned elements may activate users’ personal values such as family, friendship, freedom, and so on. However, we believe that user-oriented worth elements may concern different sensitivities depending on factors such as the application domain, the targeted users group, etc.

  • #2. Worth is twofold: appreciated vs. requested

During interviews, participants mentioned both the positive and negative sides of Cocoon. By positive, we mean what would motivate them to buy, learn, use, or recommend the system, such as diversity of types of information. We call this positive side: the “appreciated worth”. By negative we mean what would discourage them from buying, learning, using, or recommending Cocoon, such as information overload. We call elements that compensate this negative side: the “requested worth”. In the case of Cocoon, user control was part of requested worth at the end of the study of needs phase.

  • #3. It is difficult to directly question about worth

VSD heuristics for interviewing stakeholders suggest asking about values both directly and indirectly [14]. The “why” questions that opportunistically came out during interviews indirectly asked about worth. Our closing question, “what is the worth of Cocoon from your point of view?”, was meant to directly ask about worth. Most participants first reacted to the question by asking “what do you mean by worth?”. This reaction was not surprising considering that worth can be related to many domains in life, whether financial or not. Then, the interviewer had to reformulate the question using the definition of worth: “What would motivate you to buy, learn, use, and recommend this system”. After reformulation, participants managed to start listing points that make Cocoon worthwhile from their point of view. Indeed, most of elements had been already mentioned. Thus, the question represented a good means for summarizing and, sometimes, clarifying statements about the system’s worth from participants point of view.

3.3 Design

In WCD, the design phase aims to design and implement the interactive product. One of the main goals of the design phase consists in demonstrating the system’s credibility, thus ensuring worth achievement. As mentioned in the state-of-the-art, in order to achieve this goal, Worth Delivery Scenarios (WoDS), Worth Maps (WMs), and impact matrices can be applied.

During the development of Cocoon, the first activity of the design phase consisted in challenging the credibility of the adapted version (on the basis of outcomes from interviews) of the system. At this stage, the main question to be addressed was: What tool(s) to apply? We chose to investigate WMs because they also consider connections among worthies and provide a visual representation of the system worth. This choice raised subsequent issues: How to start worth mapping? What tool(s) to use? We achieved an intensive literature review on WMs and related concepts (Hierarchical Value Maps (HVMs) and Means-Ends Chains (MECs)). Outcomes from this work revealed a lack of a method and/or framework supporting worth mapping. Indeed, papers related to worth mapping only show examples of WMs that are different in terms of hierarchy levels and classes of worth elements [6, 7, 9–11, 19]. To fulfill this need, we worked in collaboration with five experts involved in interactive system design (a project manager, a UI and interaction designer, a psychologist, a graphic designer, and a software engineer) and proposed the PEW (Perceived-Expected Worth) framework for supporting worth mapping [2]. This framework suggests that:

  • WMs are formed of three main categories of elements: (1) perceived worth, i.e. appreciated worth, (2) expected worth, i.e. requested worth, and (3) Native Software and Hardware Components (NSHC) of the device(s) hosting interaction;

  • WMs still follow a vertical representation and clearly separate the three main categories of elements that we propose: appreciated worth on the upper part, requested worth on the lower part, and NSHC in the middle as they may support both appreciated worth and requested worth features.

With this renaming, PEW becomes the ARROW framework (Appreciations, Requirements and Rationale Of Worth).

Outcomes from our literature review on WMs also revealed a lack of appropriate tools for WMs construction. Indeed, for instance, during our review of the WCD-related literature, we discovered LadderUXFootnote 1, an online tool supporting quantitative data analysis from laddering. However, even though it represents an interesting tool, LadderUX is not well-suited for WMs construction since it only considers classes of elements specific to MECs (attributes, consequences, and values). Yet, a major difference between HVMs and WMs lies in the refinement and extension of considered classes of elements [10].

In the absence of an appropriate tool, we proceeded to constructing Cocoon’s WMs using Microsoft PowerPoint. It is worth noting that WMs construction was also successfully achieved using Microsoft Visio [10]. Figure 3 depicts examples of excerpts of WMs constructed during the development of Cocoon at respectively design and evaluation phases and on the basis of our ARROW framework for worth mapping.

Fig. 3.
figure 3

The ARROW framework illustrated on Cocoon. The bold chains of arrows are to be read as follows: the contextual information push feature supported by the mobile device GPS presents the quality of being diverse (because serves the user with different types of information); this diversity enhances discovery of new information (e.g. related to a chateau, a museum) and results in a gain of general knowledge for the user.

Full size image

Activities carried out during the development of Cocoon in design phase also include the creation of mock-ups using the Axure softwareFootnote 2. Since Axure allows mapping desired behavior to UIs, both the system’s UI and associated interactions were illustrated through several mock-ups. It is important to note that the first WM of Cocoon, constructed at the end of the study of needs phase, supported the creation of mock-ups particularly regarding the UI design.

After the mock-ups creation completed, usability testing (an evaluation activity) was conducted. Eleven (11) persons, ranging from 16 to 38 years old (mean: 24), participated to usability testing sessions. Sessions lasted about one hour and, during each session, participants had to achieve three scenarios using the interactive mock-ups available via the Web browser of a laptop. During interaction, tasks completion times were measured and sequences of actions recorded.

Outcomes About Cocoon

Analyses of usability testing data show first that Cocoon is easy to learn. Indeed, completion times decreased for tasks which were achieved several times during the test: from 0.36 min (the first time) to 0.02 min (the last time) for consulting information and from 1.31 min to 0.7 min for sending a post. Second, usability testing showed that Cocoon is globally easy to use. Indeed, in addition to low completion times we computed for repetitive tasks, analyses only revealed misunderstandings regarding particularly one interaction paradigm (sliding buttons) and application-related terms. The system’s UI and some associated interactions were updated prior to implementation in order to solve problems highlighted by usability testing sessions.

Lessons Learned About WCD

During the design phase, we carried out several activities: development of a framework for worth mapping in collaboration with experts involved in interactive systems design, construction of an initial WM for Cocoon, mocking-up, and usability testing. Lessons that we have learned are the following.

  • #4. WMs support design in different ways

Our main expectation from WMs was to demonstrate the credibility of design. However, our collaboration with expert designers showed that WMs can also help designers as support to UI and interaction design (through (non-functional) qualities of features), support to graphic design (particularly through human-oriented elements), support to software implementation (through (functional) qualities of features). For instance, the requested quality of modularization (see Fig. 3) guided us to organize the UI dedicated to control parameters setting using blocks. According to graphic designer, he would not hesitate to use joyful colors for Cocoon as the study of worth revealed amusement as part of the user experience.

Finally, our collaboration with experts involved in interactive systems design revealed WMs as support to communication as they visually provide an overview of the system as well as information regarding different aspects.

  • #5. Users should be involved in worth mapping

One of the strength of WMs (compared to WoDS for instance) is the highlight of connections between worth elements. Yet, during interviews conducted in the study of needs phase, we did not put much attention to these connections (our focus was mainly on reaching elements that go beyond usability). As a consequence, during worth mapping sessions, even though most worth elements were clearly expressed (thanks to worth operationalization and worth enrichment), our design team had to rely on interviews transcriptions for clarifying and better understanding links among Cocoon worth elements. We believe that involvement of users in the worth mapping process can help avoid such situations. In our opinion, this can be achieved by paying more attention to possible existing connections among worth elements during study of needs or through special worth mapping sessions with users at design phase.

  • #6. WMs construction is tedious and time consuming

Experts who participated in the elaboration of our ARROW framework for worth mapping expressed concerns about constructing WMs from scratch without the support of a dedicated tool, manually or with inappropriate tools. Indeed, construction of the Cocoon WM using PowerPoint was tedious and time consuming, thus has proved it possible but still the lack of an appropriate tool could hinder use of WMs by designers.

3.4 Evaluation

In WCD, the system evaluation should start at an early stage. As related earlier, in the development of Cocoon, evaluation started at the end of the study of needs phase with worth translation (see Sect. 3.2). The evaluation process continued throughout the system development and usability testing was conducted at design phase (see Sect. 3.3). Here, we focus on the final evaluation of the system implemented on the basis of mock-ups resulting from design phase.

It is important to note that the final version of the Cocoon application was instrumented with a probe collecting actual usage data (received information, sent and received posts, access to control parameters).

Considering the requirements for evaluation in WCD, we elaborated a strategy consisting of a field study for experience with Cocoon in the real word, SUS questionnaires for a subjective assessment of the system’s (real) usability, and group interviews for feedback.

We conducted a field study of three weeks with 15 participants ranging from 19 to 31 years old (mean: 25). In order to ensure social interactions, we recruited five groups of three people knowing each other. Most participants were friends. We only had one group of colleagues (working as air-traffic controllers), one group with a couple (partners living in the same house), and one group with two cousins.

About one week after the start of field trials, we sent the SUS questionnaire [1] to participants. After field trials ended, we conducted interviews with the five groups of participants. All group interviews were video-recorded.

Data collected during evaluation was analyzed using different techniques. Participants’ discourses were transcribed and analyzed similarly as in the study of needs phase (see Sect. 3.2). The subjective usability score was computed from SUS questionnaires and various statistical treatments were applied to actual usage data.

Outcomes from evaluation data analysis allowed us understanding and highlighting worth of Cocoon. However, a main issue remained unresolved: How to assess worth achievement in terms of the 5Ds as suggested in the WCD literature?

Our review of the WCD literature also revealed the lack of a framework and/or method supporting assessment of worth achievement according to the 5Ds. In order to fulfill this need, we engaged in the development of a method for worth assessment. Giving their potential, we investigated WMs for this purpose. Indeed, WMs based on the ARROW framework are well suited for supporting worth evaluation because the framework proposes that WMs capture the system’s global state by considering both perceived positive and missing elements, but clearly separated, and associations between them.

Our approach suggests comparing different WMs, constructed at different stages, in order to trace the evolution of the system. Then, the goal of the design is to transform requested worth to appreciated worth. Taking our framework for worth mapping as the basis for reasoning:

  • Worth is donated (the system delivers more than the worth that was intended) when additional appreciated worth elements appear, from a preceding design stage to a current design stage, and when there is no requested worth;

  • Worth is delivered (the system delivers as much as the worth that was intended), from a preceding design stage to a current design stage, when the current WM has no requested worth;

  • Worth is degraded (the system delivers less than the worth that was intended), from a preceding design stage to a current design stage, when requested worth element(s) remain in the current WM;

  • Worth is destroyed from a preceding design stage to a current design stage, when appreciated worth elements in the preceding WM turn to be part of requested worth in the current WM. Note, we define Destruction as of a higher priority than Degrading (i.e., worth is considered as destroyed whenever at least one appreciated worth element turns to be part of requested worth).

Following our approach to worth assessment, we constructed a second WM and compared it to the first one constructed at the end of the study of needs phase. Findings from evaluation of our final system and lessons learned from this phase are summarized below.

Outcomes About Cocoon

  • Cocoon represents a worthwhile system with regard to different aspects

Actual usage data revealed that participants intensively used Cocoon. Except the universe of worth “Emotions and Feelings”, all the other universes of worth identified at that study of needs phase surface from analyses: “Discovery and Diversity”, “Exchange and Communication”, and “Adaptation and Presentation of Information”. Indeed, during interviews, participants declared having learned things about their Cocoon contacts and surroundings. Participants also appreciated the automatic push, adaptation of information according to context, and posts which, according to them (and particularly the group of colleagues), incited them to communicate more.

During interviews, participants also declared that separation of the personal and impersonal information in the UI by two different icons was very helpful since it allows determining the types of received information at notification time.

  • System’s control is requested but not used

Even though user control appeared as a must during the study of needs, surprisingly, usage data showed that participants accessed to control parameters only a few times. Yet, during interviews, we did not receive complains regarding information overload and redundancy or intrusion in general. In our opinion, this can be explained by the fact that Cocoon relies on a sophisticated information filtering algorithm that also considers time frames between information deliveries and favors diversity of delivered information. This outcome shows that it is, indeed, necessary to provide users with control means but that the system should, by default, take over a part of control.

  • Cocoon delivers worth in the real world

Figure 3 compares the WM constructed at design and evaluation phases. We can notice that the second WM has no requested worth. As a consequence, according to our approach to worth assessment, Cocoon delivered (Delivery) the intended worth in the real word. However, a closer analysis shows that this worth delivery is only partial since some appreciated worth elements of the first WM are not present in the second one. In our opinion, absence of deep social ties between participants and the relatively short evaluation period may explain this absence of some elements in the second WM.

Lessons Learned About WCD

  • #7. Not all worth element is measurable

The first evaluation task of WCD – i.e., worth operationalization – suggests translating worth elements into measurable criteria. Unsurprisingly and as our experience confirmed, not all worth element (particularly human-oriented ones) can be (objectively) measured. Our method for worth assessment allows overcoming this issue since its application does not require that worth elements are necessarily measurable.

  • #8. WMs comparison favors attention to appreciated worth

Usually, focus is put on negative elements during design. Yet, changes may impact positive elements as well. Since WMs consider both the positive and the negative, changes in appreciated worth elements also attract attention.

  • #9. WMs comparison is complex

Our approach suggests comparing WMs constructed at different design times. In the absence of an appropriate tool supporting such a task, we proceeded to a paper-based WMs comparison. However, even though WMs of Cocoon were relatively small, this task was difficult to achieve. Therefore, WMs comparison would probably be unmanageable by humans in most actual design settings.

4 Discussion and Research Agenda

This paper relates the worth-centered design of Cocoon, a context-aware and mobile application. Our work shows the applicability of WCD in an actual design setting and makes the following contributions:

  • In addition to present a compilation of the state-of-the-art in WCD together with a complete operationalization of the WCD framework, the paper provides further insights about worth and introduces the notions of appreciated vs. requested worth.

  • The paper highlights strengths and weaknesses of WCD as well as practical issues related to its operationalization through nine lessons from experience.

  • The paper makes contributions with respect to the development of tools and techniques supporting worth design and evaluation. It presents the ARROW framework for worth mapping (as chains of arrows) and an approach to worth assessment.

These contributions open discussion and perspectives.

4.1 Discussion

Based on our experience, we summarize WCD as a design philosophy that provides designers with six meta-principles, five “D”s, one framework, and a set of tools, techniques, and methods for designing interactive systems that deliver worth in the real world. The point is not to say whether outcomes from this study could be reached using another approach. The point is to provide further insights about the operationalization of WCD. Clearly the design of Cocoon has proven that WCD is appropriate for accounting for more human-oriented elements but also for system-oriented ones.

WCD meta-principles are well supported by WCD specific tools, techniques, and methods. Indeed, these meta-principles provide a particular mind state which eases communication within the design team, remains designers focused on the design objectives, encourages them to consider human-oriented criteria but also system-oriented criteria that could degrade or destroy worth, and assign importance to positive as well as negative aspects. Our design case witnesses to this last point. On the one hand, because we modeled the positive and the negative sides of Cocoon through WMs, we detected that some elements that had been perceived as worthwhile at the study of needs phase disappeared at the evaluation phase. On the other hand, outcomes from the system’s evaluation pointed out some directions for improvement (mainly minor changes of the UI). However, since evaluation did not point out any defect and/or adverse outcome that would hinder the use of the system at its current state (perceived benefits are worth the defects), iteration would rather focus on understanding reasons for disappearance of elements that had been perceived as worthwhile at the study of needs phase.

4.2 Future Work

Research agenda is fourfold.

  • A better understanding of relationships between WCD and other approaches

This paper presents further insights about worth and WCD. However, during the development of Cocoon questions related to relationships between WCD and other designs approaches, such as VSD, often rose: there is a need for a better understanding of similarities and differences between different methodologies as well as possible relationships between their underlying core notions. For this, in our opinion, reflections and investigations conjointly conducted by practitioners and researchers interested in the different topics would be significantly beneficial to the community.

  • Diffusion of WCD

So far, WCD has not received much attention. We believe further understandings about worth and WCD compared to other existing design criteria and methodologies will probably encourage worth-centered developments. However, in our opinion diffusion of WCD also represents a key point to this progress that deserves a special attention through, for instance, a dedicated platform to WCD, workshops and special sessions in conferences on the topic.

  • Improvement and development of WCD-specific tools

Many WCD-specific tools and techniques exist. During our experience, we focused on WMs. Even though our work has made contributions in regard to WMs, several challenges still need to be addressed. At the current state, the main issue is the absence of an appropriate tool supporting WMs-based development (particularly WMs construction and comparison). Further developments are also required regarding worth assessment. In the near feature, we plan to improve our approach to assessing worth achievement according to the 5 D’s and propose a computerizable version that can be integrated to relevant tools.

  • Dynamic WMs for making the adaptation of interactive systems driven by worth

This includes research on how to present WMs to end-users to provide them with control over the adaptation process.