Employing user profiles to test a new version of a GUI component in its context of use
- 107 Downloads
Advancement in reusable component technology has had a significant impact on the development of complex graphical user interfaces (GUIs), which are front-ends to most of today’s software. Software developers can, with very little effort, integrate components into their software’s GUI. Problems, however, arise when new versions of GUI components replace their predecessors in an implementation. Often, the inclusion of a new version of a component breaks some part of the software, i.e., tasks that end-users were able to perform before modifications were made can no longer be performed. Software developers (who also happen to be component users) are unable to perform adequate regression testing in this context because of several factors, including lack of source code, environmental side-effects on GUI rendering, event-driven nature of GUIs, and large number of possible permutations of events. This paper presents a new technique that end-users’ quality assurance (QA) teams can employ to test the new version of a component in its application context by using the existing version as a baseline. The technique combines lightweight event-level dynamic profiling to collect user profiles in a transparent manner, GUI reverse engineering to extract the structure of the component’s GUI, test case execution to replay the collected profiles on the new version, and GUI oracles that collect properties from the existing version. Empirical studies demonstrate the practicality, usefulness, and limitations of the technique.
KeywordGUI testing User profiles Software components Test oracles User session data Automated testing
Unable to display preview. Download preview PDF.
- Abbot Java GUI Test Framework, 2003. http://abbot.sourceforge.net.
- Bertolino, A., Polini, A. 2003. A framework for component deployment testing. In Proceedings of the 25th International Conference on Software Engineering, pp. 221–231, IEEE Computer Society.Google Scholar
- Beydeda, S., Gruhn, V. 2001. An integrated testing technique for component-based software.Google Scholar
- Capture-Replay Tool, 2003. http://soft.com/.
- Dawson, S., Jahanian, F., Mitton, T., Tung, T.-L. 1996. Testing of fault-tolerant and real-time distributed systems via protocol fault injektion. In Symposium on Fault-Tolerant Computing, pp. 404–414.Google Scholar
- Dillon, L.K., Ramakrishna, Y.S. 1996. Generating oracles from your favorite temporal logic specifications. In Proceedings of the Fourth ACM SIGSOFT Symposium on the Foundations of Software Engineering (New York), acknowledgementsol. 21 of ACM Software Engineering Notes, pp. 106–117, ACM Press.Google Scholar
- Dillon, L.K., Yu, Q. 1994. Oracles for checking temporal properties of concurrent systems. In Proceedings of the ACM SIGSOFT ’94 Symposium on the Foundations of Software Engineering, pp. 140–153.Google Scholar
- Finsterwalder, M. 2001. Automating acceptance tests for gui applications in an extreme programming environment. In Proceedings of the 2nd International Conference on eXtreme Programming and Flexible Processes in Software Engineering, pp. 114–117.Google Scholar
- Hamlet, D., Mason, D., Woit, D. 2001. Theory of software reliability based on components. In Proceedings of the 23rd International Conference on Software Engineering, pp. 361–370, IEEE Computer Society.Google Scholar
- Haraty, R.A., Mansour, N., Daou, B. 2001. Regression testing of database applications. In Proceedings of the 2001 ACM Symposium on Applied Computing, pp. 285–289, ACM Press.Google Scholar
- Harrold, M.J. 2000. Testing: a roadmap. In Proceedings of the Conference on The Future Software Engineering, pp. 61–72, ACM Press.Google Scholar
- Hicinbothom, J.H., Zachary, W.W. 1993. A tool for automatically generating transcripts of human-computer interaction. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, vol. 2 of Special Sessions: Demonstrations, p. 1042.Google Scholar
- Huang, Y.-W., Huang, S.-K., Lin, T.-P., Tsai, C.-H. 2003. Web application security assessment by fault injection and behavior monitoring. In Proceedings of the Twelfth International Conference on World Wide Web, pp. 148–159, ACM Press.Google Scholar
- JUnit, Testing Resources for Extreme Programming. http://junit.org/news/extension/gui/index.htm.
- Korel, B. 1999. Black-box understanding of COTS components. In Proceedings: Seventh International Workshop on Program Comprehension, pp. 92–99, IEEE Computer Society Press.Google Scholar
- Kranitis, N., Paschalis, A., Gizopoulos, D., Zorian, Y. 2002. Effective software self-test methodology for processor cores. In Proceedings of the Conference on Design, Automation and Test in Europe, p. 592, IEEE Computer Society.Google Scholar
- Krstic, A., Lai, W.C., Cheng, K.T., Chen, L., Dey, S. 2002. Embedded software-based self-testing for soc design. In Proceedings of the 39th Conference on Design Automation, pp. 355–360, ACM Press.Google Scholar
- Lee White, H.A., Alzeidi, N. 2001. User-based testing of gui sequences and their interactions. In Proceedings of the 12th International Symposium Software Reliability Engineering, pp. 54–63.Google Scholar
- Lu, G. 1991. Automatic testing for the applications with direct manipulation interfaces. In Proceedings of the Fourth International Conference on Human-Computer Interaction, vol. 1 of Congress II: Design and Implementation of Interactive Systems: SOFTWARE ENGINEERING—METHODS, TECHNIQUES AND TOOLS; Applications of Techniques and Tools, pp. 602–606.Google Scholar
- Marick, B. 2002. Bypassing the GUI. Software Testing and Quality Engineering Magazine: 41–47.Google Scholar
- Memon, A.M. 2001. A Comprehensive Framework for Testing Graphical User Interfaces. Ph.D. thesis, Department of Computer Science, University of Pittsburgh.Google Scholar
- Memon, A.M. 2002. GUI testing: Pitfalls and process. IEEE Computer 35(8)90–91.Google Scholar
- Memon, A.M., Banerjee, I., Nagarajan, A. 2003. GUI ripping: Reverse engineering of graphical user interfaces for testing. In Proceedings of the 10th Working Conference on Reverse Engineering.Google Scholar
- Memon, A.M., Pollack, M.E., Soffa, M.L. 1999. Using a goal-driven approach to generate test cases for GUIs. In Proceedings of the 21st International Conference on Software Engineering, pp. 257–266, ACM Press.Google Scholar
- Memon, A.M., Pollack, M.E., Soffa, M.L. 2000a. Automated test oracles for GUIs. In Proceedings of the ACM SIGSOFT 8th International Symposium on the Foundations of Software Engineering (FSE-8) (NY), pp. 30–39.Google Scholar
- Memon, A.M., Pollack, M.E., Soffa, M.L. 2000b. Plan generation for GUI testing. In Proceedings of the Fifth International Conference on Artificial Intelligence Planning and Scheduling, pp. 226–235, AAAI Press.Google Scholar
- Memon, A.M., Soffa, M.L. 2003. Regression testing of guis. In Proceedings of the 9th European Software Engineering Conference held jointly with 10th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 118–127, ACM Press.Google Scholar
- Memon, A.M., Soffa, M.L., Pollack, M.E. 2001b. Coverage criteria for GUI testing. In Proceedings of the 8th European Software Engineering Conference (ESEC) and 9th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE-9), pp. 256–267.Google Scholar
- Memon, A.M., Xie, Q. 2004. Using transient/persistent errors to develop automated test oracles for event-driven software. In Proceedings of The International Conference on Automated Software Engineering 2004 (ASE’04), pp. 186–195.Google Scholar
- Mercury Interactive WinRunner, 2003. http://www.mercuryinteractive.com/products/winrunner.
- Morris, J., Lam, P., Lee, G., Parker, K., Bundell, G.A. 2002. Determining component reliability using a testing index. In Proceedings of the Twenty-fifth Australasian Conference on Computer Science, pp. 167–176, Australian Computer Society, Inc.Google Scholar
- Myers, B.A. 1993. Why are human-computer interfaces difficult to design and implement? Technical Report CS-93-183, Carnegie Mellon University, School of Computer Science.Google Scholar
- Nagarajan, A., Memon, A.M. 2003. Refactoring using event-based profiling of GUI applications. In Proceedings of the First International Workshop on REFactoring: Achievements, Challenges, Effects 2003.Google Scholar
- Orso, A., Harrold, M.J., Rosenblum, D.S. 2001a. Component metadata for software engineering tasks. In Revised Papers from the Second International Workshop on Engineering Distributed Objects, pp. 129–144, Springer-Verlag.Google Scholar
- Orso, A., Harrold, M.J., Rosenblum, D.S., Rothermel, G., Soffa, M.L., Do, H. 2001b. Using component metadata to support the regression testing of component -based software. In Proceedings of the IEEE International Conference on Software Maintenance.Google Scholar
- Rational Robot, 2003. http://www.rational.com.ar/tools/robot.html.
- Richardson, D.J., Leif-Aha, S., OMalley, T.O. 1992. Specification-based Test Oracles for Reactive Systems. In Proceedings of the 14th International Conference on Software Engineering, pp. 105–118.Google Scholar
- Ulrich, A., Chrobok-Diening, G. 1999. International workshop on testing distributed component-based systems. SIGSOFT Software Engineering Notes 24(4):43–46.Google Scholar
- Voas, J., Payne, J. 1998. Cots software failures: Can anything be done? In Proceedings of the First IEEE Workshop on Application Specific Software Engineering and Technology (ASSET’98), pp. 140–145, IEEE Press.Google Scholar
- White, L. 1996. Regression testing of GUI event interactions. In Proceedings of the International Conference on Software Maintenance (Washington, Nov.4–8), pp. 350–358.Google Scholar
- Wittenberg, C.H. 2000. Progress in testing component-based software (abstract only). In Proceedings of the International Symposium on Software Testing and Analysis, p. 178, ACM Press.Google Scholar
- Xie, Q., Memon, A.M. 2005. Rapid crash testing for continuously evolving GUI-based software applications. In Proceedings of The International Conference on Software Maintenance 2005 (ICSM’05) (Budapest, Hungary).Google Scholar
- Xie, Q., Memon, A.M. Designing and comparing automated test oracles for GUI-based software applications. ACM Transactions on Software Testing and Methodology (to appear).Google Scholar
- Zhao, W., Papachristou, C. 1998. Testing dsp cores based on self-test programs. In Proceedings of the Conference on Design, Automation and Test in Europe, pp. 166–172, IEEE Computer Society.Google Scholar