Twenty years of value sensitive design: a review of methodological practices in VSD projects


This article reviews the academic literature (1996–2016) that emerged under value sensitive design (VSD). It investigates those VSD projects that employed the tripartite methodology, examining the use of VSD methodological elements, and illustrating common practices and identifying shortcomings. The article provides advice for VSD researchers on how to complete and enhance their methodological approach as the research community moves forward.


Value sensitive design (VSD) can look back at over 20 years of constant development and is considered by many as the most comprehensive approach to account for human values in technology design (e.g., Manders-Huits 2011; Davis and Nathan 2015). Against this background, the goal of this paper is to review the methodological practices in VSD projects. We want to understand how VSD has been applied, especially in terms of the reported use of selected methodological elements for design projects that employ the VSD tripartite methodology. To achieve successful incorporation of human values in the design process, VSD employs an integrative and iterative tripartite methodology, consisting of conceptual, empirical and technical investigations (Friedman et al. 2006). In the conceptual investigation, direct and indirect stakeholders are identified, followed by an analysis of how these could be harmed by or benefit from a new technology. Additionally, values implicated by the use of technology are identified and defined (Davis and Nathan 2015). As soon as values are identified and discussed, value tensions can emerge (Friedman et al. 2006). Under empirical investigation, qualitative and quantitative methods are employed to evaluate how stakeholders experience a technology with regard to the values they consider important (Manders-Huits 2011). One aim of the technical investigation is to combine insights from the other investigations and explore how a technology might be designed to support the values identified (Manders-Huits 2011). All three investigation types are interdependent and inform each other (Manders-Huits 2011). Burmeister (2016) shows how an identified value can be refined by empirical insights, which support its in-depth conceptualization and understanding. Due to the interdependency of the three investigations, we consider iterations between them as an important cornerstone of the VSD tripartite methodology.


We conducted a literature review on VSD from 1996 to 2016, using the online libraries of the Association for Computing Machinery (ACM), the Institute of Electrical and Electronics Engineers (IEEE) as well as Science Direct and Springer. We considered all peer reviewed journal publications and conference proceedings with the term “value sensitive design”, “value-sensitive design” or “vsd” in the title, abstract or as a keyword. Furthermore, we included all papers that are listed at, a website maintained by the Value Sensitive Design Research Lab and Batya Friedman, the pioneer of VSD (Friedman 1996).

We identified 229 contributions in total, of which ten were not accessible. When examining the 219 accessible papers, we found that 106 are grounded in related research domains such as ethical computing or responsible innovation and do not apply any investigation of the tripartite methodology. This left us with 113 papers, of which 63 focus solely on the conceptualization of various values, 22 contribute to individual methodological techniques of VSD (e.g.: Friedman and Hendry 2012) and 11 propose extending views (e.g.: Le Dantec et al. 2009).

Our analysis is focused on the remaining 17 publications that develop technology with the complete tripartite methodology. To review them methodologically, we consider five VSD outcomes: (1) identification of direct and indirect stakeholders; (2) identification and conceptualization of values; (3) understanding of value harms and benefits; (4) development of mitigation strategies for value tensions; and (5) presentation of technical measures to address values. In addition to these five essential outcomes, we consider quality of method description and therefore emphasize the importance of reproducibility.


Table 1 shows that VSD projects have been conducted in a wide range of technology domains, suggesting its applicability across contexts. Of the 17 projects, 16 start with a conceptual value investigation, followed by nine with an empirical analysis. Seven go right into technical investigation after conceptualization and later validate values empirically. Only four of the projects report iteration between investigations.

Table 1 Considered projects and their investigation procedure

Conceptual investigation

Three of the 17 projects mention a specific or reproducible method for stakeholder identification: Rector et al. (2015) had a brainstorming session, Boyd et al. (2016) reviewed the literature and conducted interviews, and Miller et al. (2007) identified stakeholders according to their interaction roles with the proposed technology. Only seven projects reported the demographic data of their participating stakeholders. Nine studies explicitly mention the identification of possible harm and benefits during this phase. Identification of values relied on a mixture of methods such as literature reviews from earlier technology implementations, content analysis, guided brainstorming techniques, etc. (Azenkot et al. 2011). The depth of value conceptualization varies heavily between studies and we were unable to clearly discern how conceptualization was achieved methodologically in any of the VSD projects.

Empirical investigation

Empirically, three projects employed semi-structured interviews and three used a survey. Four combined these to a mixed-method approach. The other seven studies used rich creative formats to inform their design thinking, including value scenarios (Czeskis et al. 2010; Epstein et al. 2013), prototypes (Friedman et al. 2002), focus groups (Walton and DeRenzi 2009), field experiments (Dahl and Holbø 2012), co-design workshops (Azenkot et al. 2011) or content analysis (Brush and Borning 2005). Unfortunately, only 11 studies reported on their methods in a reproducible way.

Technical investigation

All projects provided technical measures to address the identified values. 11 reviewed existing technology, connecting known technical features with the values to be addressed (i.e. encryption for security). In doing so, they relied mostly on the background knowledge of their authors, but in some cases additionally considered input from stakeholders, such as customer product reviews (Bleumers et al. 2015), rankings (Boyd et al. 2016) or semi-structured interviews (Davis 2008). The other six projects utilized stakeholder collaboration, envisioning, brainstorming or the value dams and flows method as a way to derive necessary technical requirements (Miller et al. 2007). Seven projects explicitly recognised the possibility of value tensions, but only one structured and reproducible method was proposed to deal with them—notably the value dams and flows method mentioned above. In the remaining six projects it is unclear how recognized value tensions were resolved. It is mostly the case that authors let stakeholders envision or brainstorm alternative technologies as a solution. Figure 1 summarizes our findings.

Fig. 1

Percentage of 17 VSD projects containing selected methodological elements to achieve core outcomes


Keeping in mind that conceptual, empirical and technical investigations are interdependent and that iteration is the bond between them, it is alarming that only four projects reported iterations that promise enhanced designs. VSD does not prescribe an order for the three investigations (Watkins et al. 2013). Yet, if iterations cannot be made (for example, for lack of time), then it seems reasonable to have an empirical analysis that follows upon the conceptual work in order to deepen value insights. It is only then that a technical investigation would seem advisable.

Considering technological complexity and ubiquity, identifying stakeholders (especially indirect ones) is increasingly challenging. Many studies did not employ a good methodical approach for stakeholder identification. As a result, only broadly defined stakeholder groups were used. This practice can undermine value sensitivity as the accidental exclusion of stakeholders and their values can infringe other relevant values (Manders-Huits 2011).

A key factor in advancing any scientific field is a good description of methods, as this allows for the replication and advancement of methods. This emphasizes the importance of good documentation (see Dahl and Holbø 2012) for the future of VSD. There is limited guidance on how to accomplish certain tasks (Burmeister 2016), which can be especially challenging for researchers new to the field. A detailed report on how value conceptualization is achieved methodologically is particularly rare (see: Rector et al. 2015). A good overview of available VSD methods can be found in Friedman et al. (2017).

In order to derive technical design solutions, most authors used known features from existing technology. Our suggestion is to use a more structured approach, such as the value dams and flow method (Miller et al. 2007), to determine whether such features are in conflict with each other. It also seems promising to explore other approaches for VSD that have been proposed in the classical field of requirement engineering to support human value.


VSD has generated a rich body of literature on value conceptualizations, methodological papers and projects. Despite this success, we see a lack of methodological guidance, especially for researchers new to VSD. There is room for experienced VSD researchers to provide more best practices, which are evaluated on their long-term effects. There is also the need for more reproducible guidance and methodological descriptions to ease entrance barriers to the field. Lowering method barriers in combination with the existing openness of VSD towards other disciplines and methods is a fertile ground for evolving the discipline and its relevance over the next 20 years.


  1. Azenkot, S., Prasain, S., Borning, A., Fortuna, E., Ladner, R. E., & Wobbrock, J. O. (2011). Enhancing independence and safety for blind and deaf-blind public transit riders. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 3247–3256). ACM.

  2. Bleumers, L., Mouws, K., Huyghe, J., Van Mechelen, M., Mariën, I., & Zaman, B. (2015). Sensitivity to parental play beliefs and mediation in young children’s hybrid play activities. In Proceedings of the 14th international conference on interaction design and children (pp. 170–177). ACM.

  3. Boyd, K., Rule, A., Tabard, A., & Hollan, J. (2016). Sharing, human values, and computer activity tracking. In Proceedings of the 19th ACM conference on computer supported cooperative work and social computing companion (pp. 233–236). ACM.

  4. Brush, A. B., & Borning, A. (2005). ‘Today’messages: Lightweight support for small group awareness via email. In Proceedings of the 38th annual Hawaii international conference on system sciences (pp. 16a–16a). IEEE.

  5. Burmeister, O. K. (2016). The development of assistive dementia technology that accounts for the values of those affected by its use. Ethics and Information Technology, 18, 185–198.

    Article  Google Scholar 

  6. Cummings, M. L. (2006). Integrating ethics in design through the value-sensitive design approach. Science and Engineering Ethics, 12(4), 701–715.

    Article  Google Scholar 

  7. Czeskis, A., Dermendjieva, I., Yapit, H., Borning, A., Friedman, B., Gill, B., & Kohno, T. (2010). Parenting from the pocket: Value tensions and technical directions for secure and private parent-teen mobile safety. In Proceedings of the sixth symposium on usable privacy and security. ACM.

  8. Dahl, Y., & Holbø, K. (2012). Value biases of sensor-based assistive technology: Case study of a GPS tracking system used in dementia care. In Proceedings of the designing interactive systems conference (pp. 572–581). ACM.

  9. Davis, J. (2008). Engaging and informing citizens with household indicators. In Proceedings of the 41st annual Hawaii international conference on system sciences (pp. 190–190). IEEE.

  10. Davis, J., & Nathan, L. P. (2015). Value sensitive design: Applications, adaptations, and critiques. In Handbook of ethics, values, and technological design: Sources, theory, values and application domains (pp. 11–40). Dordrecht: Springer.

    Google Scholar 

  11. Epstein, D. A., Borning, A., & Fogarty, J. (2013). Fine-grained sharing of sensed physical activity: A value sensitive approach. In Proceedings of the 2013 ACM international joint conference on pervasive and ubiquitous computing (pp. 489–498). ACM.

  12. Friedman, B. (1996). Value-sensitive design. Interactions, 3(6), 16–23.

    Article  Google Scholar 

  13. Friedman, B., & Hendry, D. (2012). The envisioning cards: A toolkit for catalyzing humanistic and technical imaginations. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1145–1148). ACM.

  14. Friedman, B., Hendry, D. G., & Borning, A. (2017). A survey of value sensitive design methods. Foundations and Trends® in Human-Computer Interaction, 11(2), 63–125.

    Article  Google Scholar 

  15. Friedman, B., Howe, D. C., & Felten, E. (2002). Informed consent in the Mozilla browser: Implementing value-sensitive design. In Proceedings of the 35th annual Hawaii international conference on system sciences, 2002. HICSS. IEEE.

  16. Friedman, B., Kahn, P., & Borning, A. (2006). Human-computer interaction in management information systems: Foundations, chapter value sensitive design and information systems (pp. 348–372). New York: Armonk.

    Google Scholar 

  17. Kuznetsov, S. (2006). Motivations of contributors to Wikipedia. ACM SIGCAS Computers and Society, 36(2), 1.

    Article  Google Scholar 

  18. Le Dantec, C. A., Poole, E. S., & Wyche, S. P. (2009). Values as lived experience: Evolving value sensitive design in support of value discovery. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1141–1150). ACM.

  19. Manders-Huits, N. (2011). What values in design? The challenge of incorporating moral values into design. Science and Engineering Ethics, 17(2), 271–287.

    Article  Google Scholar 

  20. Miller, J. K., Friedman, B., Jancke, G., & Gill, B. (2007). Value tensions in design: The value sensitive design, development, and appropriation of a corporation’s groupware system. In Proceedings of the 2007 international ACM conference on supporting group work (pp. 281–290). ACM.

  21. Rector, K., Milne, L., Ladner, R. E., Friedman, B., & Kientz, J. A. (2015). Exploring the opportunities and challenges with exercise technologies for people who are blind or low-vision. In Proceedings of the 17th international ACM SIGACCESS conference on computers & accessibility (pp. 203–214). ACM.

  22. Schwartzman, Y., & Borning, A. (2007). The indicator browser: A web-based interface for visualizing UrbanSim simulation results. In 40th Annual Hawaii international conference on system sciences, 2007. HICSS 2007 (pp. 92–92). IEEE.

  23. Walton, R., & DeRenzi, B. (2009). Value-sensitive design and health care in Africa. IEEE Transactions on Professional Communication, 52(4), 346–358.

    Article  Google Scholar 

  24. Watkins, K. E., Borning, A., Rutherford, G. S., Ferris, B., & Gill, B. (2013). Attitudes of bus operators towards real-time transit information tools. Transportation, 40(5), 961–980.

    Article  Google Scholar 

  25. Xu, H., Crossler, R. E., & BéLanger, F. (2012). A value sensitive design investigation of privacy enhancing tools in web browsers. Decision Support Systems, 54(1), 424–433.

    Article  Google Scholar 

Download references


Open access funding provided by Vienna University of Economics and Business (WU).

Author information



Corresponding author

Correspondence to Till Winkler.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Winkler, T., Spiekermann, S. Twenty years of value sensitive design: a review of methodological practices in VSD projects. Ethics Inf Technol (2018).

Download citation


  • Value sensitive design (VSD)
  • Literature review
  • Tripartite methodology