Skip to main content

An experiment on the impact of transparency on the effectiveness of requirements documents

Empirical Software Engineering volume 21, pages 1035–1066 (2016)Cite this article

Abstract

Effective communication is important to successful software development, but it is difficult to achieve. We believe transparency — the visibility of information to stakeholders — is an important factor in the effectiveness of communication in software projects. We theorise that more effective communication results from more transparent requirements documents. To test our theory, we conducted an experiment. We developed an operational definition of transparency with three attributes: accessibility, understandability, and relevance. We had students and software practitioners use requirements documents of differing levels of transparency based on these attributes to answer questions. We found that participants with the more transparent document spent less time, answered more questions correctly, and were more confident about their answers, than participants with the less transparent document. The results of our experiment provide evidence that our view of transparency may help evaluate the effectiveness of documents as a form of communication. Further work is needed to reproduce our results, and to determine whether they are generalizable to other types of stakeholders and forms of communication.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. This is available from http://goo.gl/sWgW2Q with the experimental materials.

  2. These are available from http://goo.gl/sWgW2Q

References

  • Al-Rawas A, Easterbrook S (1996) Communication problems in requirements engineering: A field study. In: Proceedings of Conference on Professional on Awareness in Software Engineering, pp 47–60

  • Allaby M (ed) (2008) A Dictionary of Earth Sciences. Oxford University Press

  • Anda B, Sjøberg D, Jørgensen M (2001) ECOOP 2001 Object-Oriented Programming, vol 2072. Lecture Notes in Computer Science. Springer, Berlin Heidelberg, pp 402–428

  • Awad N, Krishnan M (2006) The personalization privacy paradox: An empirical evaluation of information transparency and the willingness to be profiled online for personalization. MIS Q 30(1):13–28

    Google Scholar 

  • Bevan N, Azuma M (1997) Quality in use: Incorporating human factors into the software engineering lifecycle. In: Software Engineering Standards Symposium and Forum, 1997. Emerging International Standards. ISESS 97. 3rd IEEE International, pp 169–179

  • Bickerstaff K, Tolley R, Walker G (2002) Transport planning and participation: The rhetoric and realities of public involvement. J Trans Geogr 10(1):61–73

    Article  Google Scholar 

  • Bird C (2005) Top 10 tips for better agile. Inf Prof 2(6):33–36

    Article  Google Scholar 

  • Cerri S (2000) Effective communication skills for engineers. In: Proceedings of the 2000 IEEE Engineering Management Society, pp 625–629

  • Clarke R (1999) Internet privacy concerns confirm the case for intervention. Commun ACM 42(2):60–67

    Article  Google Scholar 

  • Coughlan J, Macredie R (2002) Effective communication in requirements elicitation: A comparison of methodologies. Requir Eng 7(2):47–60

    Article  Google Scholar 

  • Curtis B, Krasner H, Iscoe N (1988) A field study of the software design process for large systems. Commun ACM 31(11):1268–1287

    Article  Google Scholar 

  • Fleischmann K, Wallace W (2005) A covenant with transparency: Opening the black box of models. Commun ACM 48(5):93–97

    Article  Google Scholar 

  • Fleischmann K, Wallace W (2009) Ensuring transparency in computational modeling. Commun ACM 52(3):131–134

    Article  Google Scholar 

  • Forward A, Lethbridge TC (2002) The relevance of software documentation, tools and technologies: A survey. In: Proceedings of the 2002 ACM symposium on Document engineering. ACM, pp 26–33

  • Fowler M (2004) UML distilled: A brief guide to the standard object modeling language. Addison-Wesley Professional

  • Hartwick J, Barki H (2001) Communication as a dimension of user participation. IEEE Trans Prof Commun 44(1):21–36

    Article  Google Scholar 

  • Heijstek W, Kuhne T, Chaudron MR (2011) Experimental analysis of textual and graphical representations for software architecture design. In: International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, pp 167–176

  • Herlea Damian D, Eberlein A, Shaw M, Gaines B (2000) Using different communication media in requirements negotiation. IEEE Softw 17(3):28–36

    Article  Google Scholar 

  • Hertzum M, Pejtersen A (2000) The information-seeking practices of engineers: Searching for documents as well as for people. Inf Process Manag 36(5):761–778

    Article  Google Scholar 

  • Ingalls P, Frever T (2009) Growing an agile culture from value seeds. In: Agile Conference, 2009. AGILE ’09, pp 119–124

  • Jedlitschka A, Ciolkowski M, Pfahl D (2008) Reporting experiments in software engineering. In: Shull F, Singer J, Sjøberg D (eds) Guide to Advanced Empirical Software Engineering. Springer, London, pp 201–228

    Chapter  Google Scholar 

  • Leffingwell D, Widrig D (2000) Managing Software Requirements: A Unified Approach. Addison-Wesley Professional

  • Lyytinen K, Hirschheim R (1987) Information systems failures: A survey and classification of the empirical literature. Oxford University Press, Inc, pp 257–309

  • Norman G (2010) Likert scales, levels of measurement and the “laws” of statistics. Adv Health Sci Educ 15(5):625–632

    Article  Google Scholar 

  • Oliver R (2004) What is transparency?. McGraw-Hill

  • Poole W (2003) The softer side of custom software development: Working with the other players. In: Proceedings. 16th Conference on Software Engineering Education and Training, 2003. (CSEE T 2003), pp 14–21

  • Rowe G, Frewer L (2000) Public participation methods: A framework for evaluation. Sci, Technol, Hum Values 25(1):3–29

    Article  Google Scholar 

  • Saiedian H, Dale R (2000) Requirements engineering: Making the connection between the software developer and customer. Inf Softw Technol 42(6):419–428

    Article  Google Scholar 

  • Santana A, Wood D (2009) Transparency and social responsibility issues for wikipedia. Ethics Inf Technol 11:133–144

    Article  Google Scholar 

  • Schwaber K, Sutherland J (2012) The scrum guide. http://www.scrum.org/Portals/0/Documents/Scrum%20Guides/Scrum_Guide.pdf.

  • Tan B, Smith H, Keil M, Montealegre R (2003) Reporting bad news about software projects: Impact of organizational climate and information asymmetry in an individualistic and a collectivistic culture. IEEE Trans Eng Manag 50(1):64–77

    Article  Google Scholar 

  • Trochim WM (2001) Research methods knowledge base. Atomic Dog Pub., Cincinnati, OH.

  • Tu Y (2014) Transparency in Software Engineering. PhD thesis, University of Auckland, New Zealand

  • Tu Y, Tempero E, Thomborson C (2014) Evaluating presentation of requirements documents: Results of an experiment. In: Zowghi D, Jin Z (eds) Requirements Engineering, vol 432. Communications and Information Science. Springer, Berlin Heidelberg, pp 120–134

  • Verner J, Overmyer S, McCain K (1999) In the 25 years since the mythical man-month what have we learned about project management? Inf Softw Technol 41(14):1021–1026

    Article  Google Scholar 

  • Walz D, Elam J, Curtis B (1993) Inside a software design team: Knowledge acquisition, sharing, and integration. Commun ACM 36(10):63–77

    Article  Google Scholar 

  • Wohlin C, Runeson P, Höst M, Ohlsson M, Regnell B (2000) Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, MA, USA

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, The University of Auckland, Auckland, New Zealand

    Yu-Cheng Tu, Ewan Tempero & Clark Thomborson

Authors
  1. Yu-Cheng Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ewan Tempero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clark Thomborson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu-Cheng Tu.

Additional information

Communicated by: Daniela Damian

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tu, YC., Tempero, E. & Thomborson, C. An experiment on the impact of transparency on the effectiveness of requirements documents. Empir Software Eng 21, 1035–1066 (2016). https://doi.org/10.1007/s10664-015-9374-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-015-9374-8

Keywords

  • Controlled experiments
  • Functional requirements
  • Requirements specification
  • Use case models
  • Transparency
  • Communication