Annals of Software Engineering

, Volume 8, Issue 1–4, pp 289–311 | Cite as

Test and evaluation of distributed information system networks

  • Raymond A. Paul
  • Jaideep Srivastava
  • Duminda Wijesekera
Article

Abstract

As information services become increasingly ubiquitous, and are being charged for, users are demanding the ability to choose the quality of an information service based on its cost. In this environment, delivering the right information to the right user at the right time, and with appropriate quality, is the challenge. In addition, resource optimizations must be performed to the degree possible. In this paper we introduce the concept of information quality based system evaluation. The key contribution is a framework in which user/application needs for information quality can be quantified in terms of rigorously defined metrics, existing systems can be evaluated against these metrics, and the tradeoffs between information quality and its cost of provisioning can be examined. Further, this provides an approach to developing information quality aware policies and mechanisms for the system. This framework has been partially validated by applying it to the provisioning of continuous media services, i.e., audio and video, in a distributed environment. This paper presents the case study in detail. Finally, the paper outlines a research program in the area of information quality based system evaluation.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Boehm, B. (1978), Characteristics of Software Quality, American Elsevier, New York.Google Scholar
  2. Chee, C.L., C.V. Ramamoorthy and R.A. Paul (1994), “RiskExpert: An Expert STEP Metrics Analyzer,” IEEE COMPSAC, Taiwan.Google Scholar
  3. Grady, R. (1992), Practical Software Metrics for Project Management and Process Improvement, Prentice-Hall, Englewood Cliffs, NJ.Google Scholar
  4. Grady, R. (1994), “Successfully Applying Software Metrics,” IEEE Computer, September.Google Scholar
  5. Lawrence, T.F. (1997), “Metrics for Quality of Service (QoS),” Rome Laboratory working paper.Google Scholar
  6. Mayer-Patel, K. and L. Rowe (1997), “Design and Performance of the Berkeley Continuous Media Toolkit (CMT),” In Proceedings of Multimedia Computing and Networking, pp. 194–206.Google Scholar
  7. Oppenheim, A.N. (1983), Questionnaire Design and Attitude Measurement, Heinemann.Google Scholar
  8. Parikh, S.S. (1997), “StreamGroups: A Mechanism for the Synchronized Delivery of Continuous Media Streams,” M.S. project, Computer Science and Engineering, University of Minnesota, December.Google Scholar
  9. Paul, R.A. (1998), “Metrics Based Software Project Management,” Ph.D. thesis, Computer Science Department, University of Tokyo, Japan.Google Scholar
  10. Part-Enander, E., A. Sjoberg, B. Melin, and P. Isaksson (1996), The Matlab Handbook, Addison-Wesley, Reading, MA.Google Scholar
  11. Porter, A. and R.W. Selby (1990), “Empirically Guided Software Development Using Metric-Based Classification Trees,” IEEE Software, March.Google Scholar
  12. Ramanathan, S. and P.V. Rangan (1993), “Feedback Techniques for ultra-Media Continuity and Inter-Media Synchronization in Distributed Multimedia Systems,” The Computer Journal 36,1, 19–33.CrossRefGoogle Scholar
  13. Shinagawa, Y. (1995), “Multiresolution Analysis of Metric Data for Software Engineering,” Internal Technical Report, Purdue University.Google Scholar
  14. Smith, B., L.A. Rowe and S. Yen (1993), “Tcl Distributed Programming,” In Proceedings of the 1993 Tcl/Tk Workshop.Google Scholar
  15. Smith, B. (1996), “Cyclic-UDP: A New Protocol for Continuous Media Communication,” http://www.cs.cornell.edu/Info/Faculty/Brian_Smith.html/Publications.Google Scholar
  16. Sreenan, CJ. (1992), “Synchronization Services for Digital Continuous Media,” Ph.D. thesis, University of Cambridge, UK, October 1992.Google Scholar
  17. Steinmetz, R., N.D. Georganas, and T. Nakagawa (1996), “Guest Editorial: Synchronization Issues in Multimedia Communications,” IEEE Journal on Selected Areas in Communication 14,1 1–4.CrossRefGoogle Scholar
  18. Vogel, A., B. Kerherve, G. v. Bochmann, and J. Gecsei (1995), “Distributed Multimedia and QoS: A Survey,” IEEE Multimedia Magazine 3, 10–19.CrossRefGoogle Scholar
  19. Weller, E. (1994), “Using Metrics to Manage Software Projects,” IEEE Computer, 27–33.Google Scholar
  20. Wijesekera, D. and J. Srivastava (1996), “Quality of Service Metrics for Continuous Media,” International Journal on Multimedia Tools and Applications 3,1, 127–166.CrossRefGoogle Scholar
  21. Wijesekera, D. (1997), “Quality of Service for Continuous Media: Metrics, Validation, Evaluation, and Mechanisms,” Ph.D. dissertation, Computer Science & Engineering, University of Minnesota, December.Google Scholar
  22. Wijesekera, D., S. Varadarajan, S.S. Parikh, and J. Srivastava (1998), “Quality of Service (QoS) Based Evaluation of the Berkeley Continuous Media Toolkit (CMT),” submitted to the ACM/Springer-Verlag Multimedia Systems Journal.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Raymond A. Paul
    • 1
  • Jaideep Srivastava
    • 2
  • Duminda Wijesekera
    • 3
  1. 1.Office of the Secretary of Defense (Acquisition & Technology)WashingtonUSA
  2. 2.Department of Computer Science and EngineeringUniversity of MinnesotaMinneapolisUSA
  3. 3.Honeywell Space Systems, Satellite & Adv. Proc. AvionicsClearwaterUSA

Personalised recommendations