Abstract
We investigate the evolution of a medium sized software package, LA-PACK, through its public releases over the last six years and establish a correlation, at a subprogram level, between a simply computable software metric value and the number of coding errors detected in the released routines. We also quantify the code changes made between issues of the package and attempt to categorize the reasons for these changes.
We then consider the testing strategy used with LAPACK. Currently this consists of a large number of mainly self-checking driver programs along with sets of configuration files. These suites of test codes run a very large number of test cases and consume significant amounts of cpu time. We attempt to quantify how successful this testing strategy is from the viewpoint of the coverage of the executable statements within the routines being tested.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchov, and D. Sorensen. LAPACK: users' guide. SIAM, Philadelphia, second edition, 1995.
ANSI. Programming Language Fortran X3.9-1978. American National Standards Institute, New York, 1979.
L.S. Blackford, J. Choi, A. Cleary, E. D'Azevedo, J. Demmel, I. Dhillon, J. Don-garra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R.C. Whaley. ScaLAPACK Users' Guide. SIAM, Philadelphia, 1997.
David Barnes, Mark Russell, and Mark Wheadon. Developing and adapting UNIX tools for workstations. In Autumn 1988 Conference Proceedings, pages 321-333. European UNIX systems User Group, October 1988.
J. Demmel and A. McKenney. A test matrix generation suite. Technical Report MCS-P69-0389, Argonne National Laboratories, Illinois, March 1989.
J. J. Dongarra, J. Du Croz, I. S. Duff, and S. Hammarling. Algorithm 679: A set of level 3 basic linear algebra subprograms. ACM Trans. Math. Softw., 16(1):18-28, March 1990.
J. J. Dongarra, J. Du Croz, S. Hammarling, and R. J. Hanson. Algorithm 656: An extended set of basic linear algebra subprograms: Model implementation and test programs. ACM Trans. Math. Softw., 14(1):18-32, March 1988.
J. J. Dongarra, C. B. Moler, J. R. Bunch, and G. W. Stewart. LINPACK: Users' Guide. SIAM, Philadelphia, 1979.
B. S. Garbow, J. M. Boyle, J. J. Dongarra, and C. B. Moler. Matrix Eigensystem Routines - EISPACK Guide Extension, volume 51 of Lecture notes in computer science. Springer-Verlag, New York, 1977.
M.H. Halstead. Elements of Software Science. Operating and Programming Systems Series. Elsevier, New York, 1977.
Les Hatton. Does 00 sync with how we think? IEEE Software, pages 46-54, May/June 1998.
T.R. Hopkins Restructuring software: A case study. Software - Practice and Experience, 26(8):967-982, July 1996.
T.R. Hopkins Is the quality of numerical subroutine code improving? In E. Arge, A.M. Bruaset, and H.P. Langtangen, editors, Modern Software Tools for Scientific Computing, pages 311-324. Birkhäuser Verlag, Basel, 1997.
T.R. Hopkins. Restructuring the BLAS Level-1 routine for computing the modified Givens transformation. ACM SIGN UM, 32(4):2-14, October 1997.
M. Lorenz and J. Kidd. Object-Oriented Software Metrics. Object-Oriented Series. Prentice Hall, Englewood Cliffs, New Jersey, 1994.
B. A. Nejmeh. NPATH: A measure of execution path complexity and its applications. Commun. ACM, 31(2):188-200, 1988.
Numerical Algorithms Group Ltd., Oxford, UK. NAG Ware f77 Tools, second edition, September 1992.
Programming Research Ltd, Hersham, Surrey. QA Fortran 6.0, 1992.
B. T. Smith, J. M. Boyle, J. J. Dongarra, B. S. Garbow, Y. Ikebe, V. C. Klema, and C. B. Moler. Matrix Eigensystem Routines - EISPACK Guide, volume 6 of Lecture notes in computer science. Springer-Verlag, New York, second edition, 1976.
Nigel Tracey, John Clark, and Keith Mander. Automated program flaw finding using simulated annealing. In Software Engineering Notes, Proceedings of the International Symposium on Software Testing and Analysis, volume 23, pages 73-81. ACM/SIGSOFT, March 1998.
Nigel Tracey, John Clark, and Keith Mander. The way forward for unifying dynamic test-case generation: The optimisation-based approach. In International Workshop on Dependable Computing and Its Applications (DCIA), pages 169-180. IFIP, January 1998.
Nigel Tracey, John Clark, Keith Mander, and John McDermid. An automated framework for structural test-data generation. In Proceedings of the International Conference on Automated Software Engineering. IEEE, October 1998.
J. Wasniewski and J.J. Dongarra. High performance linear algebra package — LAPACK90. Technical Report CS-98-384, University of Tennessee, Knoxville, April 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Barnes, D., Hopkins, T. (2000). The Evolution and Testing of a Medium Sized Numerical Package. In: Langtangen, H.P., Bruaset, A.M., Quak, E. (eds) Advances in Software Tools for Scientific Computing. Lecture Notes in Computational Science and Engineering, vol 10. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-57172-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-57172-5_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66557-1
Online ISBN: 978-3-642-57172-5
eBook Packages: Springer Book Archive