Assessing COTS Assessment: How Much Is Enough?

  • Dan Port
  • Scott Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2959)


COTS products are now ubiquitous and clearly have become a key factor in modern software systems development. If COTS are chosen poorly, a project will likely fail. As a result, the careful assessment of COTS products has become an essential element of the development process. There are numerous approaches to COTS assessment; however none of them address the crucial question of how much assessment effort to perform. If too little assessment is done, inappropriate COTS may be used; if too much assessment is done, the effort expended may place the project at risk. It is important to achieve a satisfactory balance between COTS uncertainty risks and risks resulting from project delay. To address this, we develop a method for the strategic planning of COTS assessment by determining “how much is enough” effort (in time, cost, or quality) with respect to critical project risk factors such as project schedule, market window, and a multitude of COTS assessment attributes such as availability, ease of use, maturity, and vendor support. The method is practical, and provides valuable aid in the planning of COTS based system development.


COTS COTS assessment COTS integration software risk COTS evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Basili, V., Boehm, B.: COTS-Based Systems Top 10 List. IEEE Computer 34(5) (May 2001)Google Scholar
  2. 2.
    Ncube, C., Maiden, N.A.M.: PORE: Procurement-Oriented Requirements Engineering Method for the Component-Based Systems Engineering Development Paradigm. In: 1999 International Workshop on Component-Based Software Engineering (1999)Google Scholar
  3. 3.
    Ncube, C., Maiden, N.: COTS software selection: The need to make tradeoffs between system requirements, architectures and COTS components. In: COTS workshop. Continuing Collaborations for Successfull COTS Development (2000)Google Scholar
  4. 4.
    Alves, C., Finkelstein, A.: Challenges in COTS-Decision Making: A Goal-Driven Requirements Engineering Perspective. In: Workshop on Software Engineering Decision Support, in conjunction with SEKE 2002, Ischia, Italy (July 2002)Google Scholar
  5. 5.
    Kontio, J.: OTSO: A Systematic Process for Reusable Software Component Selection. CS-TR-3478, University of Maryland Technical Reports. University of Maryland. College Park, MD (1995)Google Scholar
  6. 6.
    Dukic, L.: Non -Functional Requirements for COTS Software Components. In: Workshop Ensuring Successful COTS Development (May 2000)Google Scholar
  7. 7.
    Dean, J.C.: Timing the Testing of COTS Software Products. In: Proceedings of the 1st Workshop on Testing Component Based Systems, Los Angeles, CA, NRC 41625 ,May 17, pp.5–8 (1999)Google Scholar
  8. 8.
    Alves, C., Castro, J.: CRE: A Systematic Method for COTS Components Selection. In: XV Brazilian Symposium on Software Engineering(SBES) Rio de Janeiro, Brazil (October 2001)Google Scholar
  9. 9.
    Tran, V., Liu, D.-B.: A Risk-Mitigating Model for the Development of Reliable and Maintainable Large-Scale Commercial-Off-The-Shelf Integrated Software Systems. In: Proceedings of the 1997 Annual Reliability and Maintainability Symposium, January 1997, pp. 361–367 (1997)Google Scholar
  10. 10.
    Jeanrenaud, J., Romanazzi, P.: Software Product Evaluation: A Methodological Approach. In: Software Quality Management II:Building Software into Quality, pp. 59–69 (1994)Google Scholar
  11. 11.
    Ochs, M., Pfahl, D., Chrobok-Diening, G., Nothelfer-Kolb, B.: A Method for Efficient Measurement-based COTS Assessment and Selection - Method Description and Evaluation Results. In: Pro. of the Seventh International Software Metrics Symposium METRICS 2001, London, April 2001, pp. 285–297 (2001)Google Scholar
  12. 12.
    Phillips, B.C., Polen, S.M.: Add Decision Analysis to Your COTS Selection Process, Software Technology Support Center Crosstalk (April 2002)Google Scholar
  13. 13.
    Boehm, B., Port, D., Yang, Y., Bhuta, J., Abts, C.: Composable Process Elements for Developing COTS-Based Applications (2002)Google Scholar
  14. 14.
    Dean, J., Vidger, M.: COTS Software Evaluation Techniques. In: Proceedings of The NATO Information Systems Technology. Symposium on Commercial Off-the-shelf Products in Defence Applications, Brussels, Belgium (April 2000)Google Scholar
  15. 15.
    Kunda, D., Brooks, L.: Applying Social-Technique approach for COTS selection. In: Proceedings of 4th UKAIS Conference, University of York, April 1999, McGraw Hill, New York (1999)Google Scholar
  16. 16.
    Gregor, S., Hutson, J., Oresky, C.: Storyboard Process to Assist in Requirements Verification and Adaptation to Capabilities Inherent in COTS. In: Dean, J., Gravel, A. (eds.) ICCBSS 2002. LNCS, vol. 2255, pp. 132–141. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  17. 17.
    Burgués, X., Estay, C., Franch, X., Pastor, J.A., Quer, C.: Combined selection of COTS components. In: Dean, J., Gravel, A. (eds.) ICCBSS 2002. LNCS, vol. 2255, pp. 54–64. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. 18.
    Comella-Dorda, S., Dean, J.C., Morris, E., Oberndorf, P.: A process for COTS Software Product Evaluation. In: Dean, J., Gravel, A. (eds.) ICCBSS 2002. LNCS, vol. 2255, pp. 86–92. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Ruhe, G.: Intelligent Support for Selection of COTS Products. In: Proceedings of the Net.ObjectDays 2002, Erfurt, Springer, Heidelberg (2003)Google Scholar
  20. 20.
    Carr, M.J., Konda, S.L., Monarch, I., Ulrich, F.C., Walker, C.F.: Taxonomy-Based Risk Identification, Software Engineering Institute, Carnegie Mellon University, Technical Report CMU/SEI-93-TR-6, ESC-TR-93-183 (June 1993)Google Scholar
  21. 21.
    Hall, E.M.: Managing Risk. Addison-Wesley Longman, Amsterdam (1998)Google Scholar
  22. 22.
    Boehm, B.: Software Risk Management: Principles and Practices. IEEE Software, 32–41 (January 1991)Google Scholar
  23. 23.
    Bertoa, M.F., Vallecillo, A.: Quality Attributes for COTS Components. In: Proc. of the 6th ECOOP Workshop on Quantitative Approaches in Object-Oriented Software Engineering (QAOOSE 2002), Málaga, Spain (June 2002)Google Scholar
  24. 24.
    Boehm, B., Abts, C., Brown, A.W., Chulani, S., Clark, B., Horowitz, E., Madachy, R., Reifer, D., Steece, B.: Software Cost Estimation with COCOMO II. Prentice Hall, Englewood Cliffs (2000)Google Scholar
  25. 25.
    Torchiano, M., Jaccheri, L.: Assessment of Reusable COTS Attributes. In: Erdogmus, H., Weng, T. (eds.) ICCBSS 2003. LNCS, vol. 2580, pp. 219–228. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  26. 26.
    Abts, C., Boehm, B., Bailey Clark, E.: COCOTS: A Software COTS-Based System (CBS) Cost Model. In: Proceedings, ESCOM 2001, April 2001, pp. 1–8 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Dan Port
    • 1
  • Scott Chen
    • 2
  1. 1.Department of Information Technology ManagementUniversity of Hawaii at ManoaHonolulu
  2. 2.Center for Software EngineeringUniversity of Southern CaliforniaLos Angeles

Personalised recommendations