Multi-threaded Testing with AOP Is Easy, and It Finds Bugs!
We investigate the suitability of AOP (Aspect Oriented Programming) for testing tools by trying to implement the ConTest testing tool using AspectJ, a tool that implements AOP for the Java programming language. We examine whether the entire set of features can be implemented this way, in the context of the larger problem where moving to a higher level of abstraction means that some details cannot be implemented.
Our conclusion from this exercise is that AOP is very suitable for the implementation of a number of classes of test tools. These include multi-threaded noise makers such as ConTest, in addition to coverage analyzers, data-race detectors, network traffic simulators, runtime bug pattern detectors, and others. The main advantage is that the instrumentation part of the tool creating method, which usually contains little scientific contribution but consumes most of the work, becomes much easier to perform and requires less expertise. In our specific exercise, a task that took more than half a year and required specialized knowledge, was reduced to two weeks work by a relative novice.
Unable to display preview. Download preview PDF.
- 1.Aspectj getting started, http://www.elipse.org/aspectj
- 2.Choi, J.-D., Srinivasan, H.: Deterministic replay of java multithreaded applications. In: Proceedings of the SIGMETRICS Symposium on Parallel and Distributed Tools (August 1998)Google Scholar
- 3.Corbett, J.C., Dwyer, M., Hatcliff, J., Pasareanu, C.: Bandera: Extracting finite-state models from Java source code. In: Proc. 22nd International Conference on Software Engineering (ICSE), June 2000, ACM Press, New York (2000)Google Scholar
- 4.Edelstein, O., Farchi, E., Goldin, E., Nir, Y., Ratsaby, G., Ur, S.: Testing multithreaded java programs. The IBM System Journal Special Issue on Software Testing (February 2002) (submitted to)Google Scholar
- 5.Edelstein, O., Farchi, E., Nir, Y., Ratsaby, G., Ur, S.: Multithreaded Java program test generation. IBM Systems Journal 41(1), 111–125 (2002), Also available as http://www.research.ibm.com/journal/sj/411/edelstein.html CrossRefGoogle Scholar
- 6.Hartman, A., Kirshin, A., Nagin, K.: A test execution environment running abstract tests for distributed software. In: Proceedings of Software Engineering and Applications, SEA (2002)Google Scholar
- 7.Havelund, K., Pressburger, T.: Model checking java programs using java pathfinder. International Journal on Software Tools for Technology Transfer, STTT 2(4) (April 2000)Google Scholar
- 8.Itzkovitz, E., Schuster, A., Zeev-Ben-Mordehai, O.: Towards integration of datarace detection in dsm systems. Journal of Parallel and Distributed Computing. Special Issue on Software Support for Distributed Computing 59(2), 180–203 (1999)Google Scholar
- 9.Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.G.: An overview of AspectJ. LNCS, vol. 2072, pp. 327–355. Springer, Heidelberg (2001)Google Scholar
- 11.Malaiya, Y., Li, N., Bieman, J., Karcich, R., Skibbe, B.: Software test coverage and reliability. Technical report, Colorado State University (1996)Google Scholar
- 12.Porat, S., Biberstein, M., Koved, L., Mendelson, B.: Automatic detection of immutable fields in java. In: Proceedings of the 2000 conference of the Centre for Advanced Studies on Collaborative research, p. 10. IBM Press (2000)Google Scholar
- 13.Richards, B., Larus, J.R.: Protocol-based data-race detection. In: Proceedings of the 2nd SIGMETRICS Symposium on Parallel and Distributed Tools (August 1998)Google Scholar
- 15.Stoller, S.D.: Model-checking multi-threaded distributed java programs. In: Proceedings of the 7th International SPIN Workshop on Model Checking (2000)Google Scholar
- 17.Stoller, S.D.: Testing concurrent java programs using randomized scheduling. In: Proceedings of the Second Workshop on Runtime Verification (RV). Electronic Notes in Theoretical Computer Science, vol. 70(4), Elsevier, Amsterdam (2002)Google Scholar