Androutsopoulos, K, Clark, D, Dan, H, Hierons, RM, Harman, M. (2014). An analysis of the relationship between conditional entropy and failed error propagation in software testing. In Proceedings of the 36th international conference on software engineering (pp. 573–583). New York: ACM.
Araujo, W, Briand, LC, Labiche, Y. (2014). On the effectiveness of contracts as test oracles in the detection and diagnosis of functional faults in concurrent object-oriented software. IEEE Transactions on Software Engineering, 40, 971–992.
Article
Google Scholar
Assi, RA, Masri, W, Zaraket, F. (2016). UCov: a user-defined coverage criterion for test case intent verification. Software Testing, Verification and Reliability, 26(6), 1–32.
Article
Google Scholar
Atkinson, C, Hummel, O, Janjic, W. (2011). Search-enhanced testing: NIER track, IEEE, Hawaii.
Baresi, L, & Young, M. (2001). Test oracles. Tech. Rep. CIS-TR-01-02, University of Oregon.
Barnett-Page, E, & Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. BMC Medical Research Methodology, 9(1), 1–26.
Article
Google Scholar
Brilliant, SS, Knight, JC, Ammann, PE. (1990). On the performance of software testing using multiple versions. In Proceedings of the 20th international symposium on fault-tolerant computing (FTCS-20) (pp. 408–415). Newcastle Upon Tyne: IEEE.
Cao, Y, Zhou, ZQ, Chen, TY. (2013). On the correlation between the effectiveness of metamorphic relations and dissimilarities of test case executions. In Proceedings of the 13th international conference on quality software (QSIC) (pp. 153–162). Najing: IEEE.
Chan, WK, Cheung, SC, Ho, JCF, Tse, TH. (2006). Reference models and automatic oracles for the testing of mesh simplification software for graphics rendering. In Proceedings of the 30th annual international computer software and applications conference (COMPSAC) (pp. 429–438). Chicago: IEEE.
Chan, WK, Cheung, SC, Leung, KRPH. (2007). A metamorphic testing approach for online testing of service-oriented software applications. International Journal of Web Services Research, 4(2), 61–81.
Article
Google Scholar
Chan, WK, Cheung, SC, Ho, JCF, Tse, TH. (2009). PAT: a pattern classification approach to automatic reference oracles for the testing of mesh simplification programs. Journal of Systems and Software, 82(3), 422–434.
Article
Google Scholar
Chan, WK, Ho, JCF, Tse, TH. (2010). Finding failures from passed test cases: improving the pattern classification approach to the testing of mesh simplification programs. Software Testing, Verification and Reliability, 20(2), 89–120.
Article
Google Scholar
Chen, TY, Cheung, SC, Yiu, SM. (1998b). Metamorphic testing: a new approach for generating next test cases. Tech. Rep. HKUST-CS98-01, Hong Kong University of Science and Technology.
Chen, TY, Tse, TH, Quan Zhou, Z. (2003b). Fault-based testing without the need of oracles. Information and Software Technology, 45(1), 1–9.
Article
Google Scholar
Chen, TY, Huang, DH, Tse, TH, Zhou, ZQ. (2004a). Case studies on the selection of useful relations in metamorphic testing. In Proceedings of the 4th Ibero-American symposium on software engineering and knowledge engineering (JIISIC) (pp. 569–583). Australia: Polytechnic University of Madrid.
Chen, J, Kuo, FC, Xie, X, Wang, L. (2014c). A cost-driven approach for metamorphic testing. Journal of Software, 9(9), 2267–2275.
Chen, TY, Ho, JWK, Liu, H, Xie, X. (2009a). An innovative approach for testing bioinformatics programs using metamorphic testing. BMC Bioinformatics, 10 (1), 1–12.
Article
Google Scholar
Chen, TY, Kuo, FC, Merkel, R, Tam, WK. (2009c). Testing an open source suite for open queuing network modelling using metamorphic testing technique. In Proceedings of the 14th IEEE international conference on engineering of complex computer systems (pp. 23–29). Potsdam: IEEE.
Chen, TY, Tse, TH, Zhou, ZQ. (2011b). Semi-proving: an integrated method for program proving, testing, and debugging. IEEE Transactions on Software Engineering, 37(1), 109–125.
Article
Google Scholar
Chen, TY, Poon, PL, Xie, X. (2016). METRIC: METamorphic relation identification based on the category-choice framework. Journal of Systems and Software, 116, 177–190.
Article
Google Scholar
Clark, D, & Hierons, RM. (2012). Squeeziness: an information theoretic measure for avoiding fault masking. Information Processing Letters, 112(8-9), 335–340.
MathSciNet
Article
Google Scholar
Cruzes, DS, & Dyba, T. (2011). Recommended steps for thematic synthesis in software engineering. In Proceedings of the international symposium on empirical software engineering and measurement (ESEM) (pp. 275–284). Alberta: IEEE.
Da Silva, FQB, Cruz, SSJO, Gouveia, TB, Capretz, LF. (2013). Using meta-ethnography to synthesize research: a worked example of the relations between personality and software team processes. In Proceedings of the ACM / IEEE international symposium on empirical software engineering and measurement (pp. 153–162). Maryland: IEEE.
Davis, MD, & Weyuker, EJ. (1981). Pseudo-oracles for non-testable programs. In Proceedings of the ACM 1981 Conference (pp. 254–257). New York: ACM.
Delamaro, ME, Nunes, FLS, Oliveira, RAP. (2013). Using concepts of content-based image retrieval to implement graphical testing oracles. Software Testing, Verification and Reliability, 23(3), 171–198.
Article
Google Scholar
Dong, G, Nie, C, Xu, B, Wang, L. (2007). An effective iterative metamorphic testing algorithm based on program path analysis. In Proceedings of the 7th international conference on quality software (QSIC) (pp. 292–297). China: IEEE.
Dong, G, Guo, T, Zhang, P. (2013). Security assurance with program path analysis and metamorphic testing. In Proceedings of the 4th IEEE international conference on software engineering and service science (ICSESS) (pp. 193–197). Beijing: IEEE.
Frounchi, K, Briand, LC, Grady, L, Labiche, Y, Subramanyan, R. (2011). Automating image segmentation verification and validation by learning test oracles. Information and Software Technology, 53(12), 1337–1348.
Article
Google Scholar
Gotlieb, A. (2003). Exploiting symmetries to test programs. In Proceedings of the 14th international symposium on software reliability engineering (ISSRE) (pp. 365–374). Colorado: IEEE.
Gotlieb, A, & Botella, B. (2003). Automated metamorphic testing. In Proceedings of the 27th annual international computer software and applications conference (COMPSAC) (pp. 34–40). Dallas: IEEE.
Guderlei, R, & Mayer, J. (2007a). Statistical metamorphic testing testing programs with random output by means of statistical hypothesis tests and metamorphic testing. In Proceedings of the 7th international conference on quality software (QSIC) (pp. 404–409). Oregon: IEEE.
Guderlei, R, & Mayer, J. (2007b). Towards automatic testing of imaging software by means of random and metamorphic testing. International Journal of Software Engineering and Knowledge Engineering, 17(6), 757–781.
Article
Google Scholar
Guderlei, R, Mayer, J, Schneckenburger, C, Fleischer, F. (2007). Testing randomized software by means of statistical hypothesis tests. In Proceedings of the 4th international workshop on software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting (pp. 46–54). New York: ACM.
Harman, M, McMinn, P, Shahbaz, M, Yoo, S. (2013). A comprehensive survey of trends in oracles for software testing. Tech. Rep. TR-09-03, University of Sheffield.
Higgins, J, & Green, S. (2011). Cochrane handbook for systematic reviews of interventions. The Cochrane Collaboration.
Hoffman, D. (1999). Heuristic test oracles. Software Testing & Quality Engineering 29–32.
Hummel, O, Atkinson, C, Brenner, D, Keklik, S. (2006). Improving testing efficiency through component harvesting. Tech. rep., University of Mannheim.
Janjic, W, Barth, F, Hummel, O, Atkinson, C. (2011). Discrepancy discovery in search-enhanced testing. In Proceedings of the 3rd international workshop on search-driven development: users, infrastructure, tools, and evaluation (pp. 21–24). New York: ACM.
Jiang, M, Chen, TY, Kuo, FC, Zhou, ZQ, Ding, Z. (2014). Testing model transformation programs using metamorphic testing. In Proceedings of the 26th international conference on software engineering and knowledge engineering (SEKE) (pp. 94–99). Vancouver: Knowledge Systems Institute Graduate School.
Kanewala, U, & Bieman, JM. (2013a). Techniques for testing scientific programs without an oracle. In Proceedings of the 5th international workshop on software engineering for computational science and engineering (SE-CSE) (pp. 48–57). California: IEEE.
Kanewala, U, & Bieman, JM. (2013b). Using machine learning techniques to detect metamorphic relations for programs without test oracles. In Proceedings of the 24th international symposium on software reliability engineering (ISSRE) (pp. 1–10). USA: IEEE.
Kanewala, U, & Bieman, JM. (2014). Testing scientific software: a systematic literature review. Information and Software Technology, 56(10), 1219–1232.
Article
Google Scholar
Kanewala, U, Bieman, JM, Ben-Hur, A. (2014). Predicting metamorphic relations for testing scientific software: a machine learning approach using graph kernels. Journal of Software Testing, Verification and Reliability 1–25.
Kitchenham, B. (2007). Guidelines for performing systematic literature reviews in software engineering. Tech. Rep. EBSE-2007-01, Keele University.
Kuo, FC, Zhou, ZQ, Ma, J, Zhang, G. (2010). Metamorphic testing of decision support systems: a case study. IET Software, 4(4), 294–301.
Article
Google Scholar
Kuo, FC, Liu, S, Chen, TY. (2011). Testing a binary space partitioning algorithm with metamorphic testing. In Proceedings of the 2011 ACM symposium on applied computing (pp. 1482–1489). New York: ACM.
Liu, H, Liu, X, Chen, TY. (2012). A new method for constructing metamorphic relations.. In Proceedings of the 12th international conference on quality software (QSIC) (pp. 59–68). Shaanxi: IEEE.
Liu, H, Kuo, FC, Towey, D, Chen, TY. (2014a). How effectively does metamorphic testing alleviate the oracle problem? IEEE Transactions on Software Engineering, 40(1), 4–22.
Article
Google Scholar
Liu, H, Yusuf, II, Schmidt, HW, Chen, TY. (2014b). Metamorphic fault tolerance: an automated and systematic methodology for fault tolerance in the absence of test oracle. In Proceedings of the 36th international conference on software engineering (pp. 420–423). New York: ACM.
Lozano, RC. (2010). Constraint programming for random testing of a trading system. Stockholm: PhD thesis Polytechnic University of Valencia.
Google Scholar
Manolache, LI, & Kourie, DG. (2001). Software testing using model programs. Software: Practice and Experience, 31(13), 1211–1236.
MATH
Google Scholar
Mayer, J. (2005). On testing image processing applications with statistical methods. In Proceedings of software engineering, lecture notes in informatics (pp. 69–78). Bonn.
Mayer, J, & Guderlei, R. (2004). Test oracles using statistical methods. In Proceedings of the 1st international workshop on software quality (pp. 179–189). Springer.
Mayer, J, & Guderlei, R. (2006). An empirical study on the selection of good metamorphic relations. In Proceedings of the 30th annual international computer software and applications conference (COMPSAC) (pp. 475–484). Chicago: IEEE.
McMinn, P. (2009). Search-based failure discovery using testability transformations to generate pseudo-oracles. In Proceedings of the 11th annual conference on genetic and evolutionary computation (pp. 1689–1696). New York: ACM.
Merkel, R, Wang, D, Lin, H, Chen, TY. (2011). Automatic verification of optimization algorithms: a case study of a quadratic assignment problem solver. International Journal of Software Engineering and Knowledge Engineering, 21 (2), 289–307.
Article
Google Scholar
Mishra, KS, Kaiser, GE, Sheth, SK. (2013). Effectiveness of teaching metamorphic testing, part II. Tech. Rep. CUCS-022-13, Columbia University.
Monisha, TR, & Chamundeswari, A. (2013). Automatic verification of test oracles in functional testing. In Proceedings of the 4th international conference on computing, communications and networking technologies (ICCCNT) (pp. 1–4). Tiruchengode: IEEE.
Murphy, C, & Kaiser, GE. (2008). Improving the dependability of machine learning applications. Tech. Rep. cucs-049-08, Columbia University, 116th and Broadway, New York, NY 10027.
Murphy, C, & Kaiser, GE. (2009). Metamorphic runtime checking of non-testable programs. Tech. Rep. cucs-042-09, Columbia University, 116th and Broadway, New York, NY 10027.
Murphy, C, & Kaiser, GE. (2010). Automatic detection of defects in applications without test oracles. Tech. Rep. CUCS-027-10, Columbia University, 116th and Broadway, New York, NY 10027.
Murphy, C, Kaiser, GE, Hu, L. (2008). Properties of machine learning applications for use in metamorphic testing. In Proceedings of the 20th international conference on software engineering and knowledge engineering (SEKE) (pp. 867–872). California: SEKE.
Murphy, C, Shen, K, Kaiser, G. (2009a). Automatic system testing of programs without test oracles. In Proceedings of the 18th international symposium on software testing and analysis (pp. 189–200). New York: ACM.
Murphy, C, Shen, K, Kaiser, G. (2009b). Using JML runtime assertion checking to automate metamorphic testing in applications without test oracles. In Proceedings of the international conference on software testing verification and validation (ICST) (pp. 436–445). Colorado: IEEE.
Murphy, C, Raunak, MS, King, A, Chen, S, Imbriano, C, Kaiser, G, Lee, I, Sokolsky, O, Clarke, L, Osterweil, L. (2011). On effective testing of health care simulation software. In Proceedings of the 3rd workshop on software engineering in health care (pp. 40–47). New York: ACM.
Murphy, C, Kaiser, GE, Bell, JS, Su, Fh. (2013). Metamorphic runtime checking of applications without test oracles. Tech. Rep. CUCS-023-13, Columbia University, 116th and Broadway, New York, NY 10027.
Nardi, PA, & Delamaro, ME. (2011). Test oracles associated with dynamical systems models. Tech. Rep. RT 362, Universidade De Sao Paulo.
Núñez, A, & Hierons, RM. (2015). A methodology for validating cloud models using metamorphic testing. Annals of Telecommunications - Annales des Télécommunications, 70(3), 127–135.
Article
Google Scholar
Oliveira, RAP, Kanewala, U, Nardi, PA. (2014a). Automated test oracles: state of the art, taxonomies, and trends. In Memon, A (Ed.) Advances in computers (pp. 113–199): Elsevier.
Oliveira, RAP, Memon, AM, Gil, VN, Nunes, FL, Delamaro, M. (2014b). An extensible framework to implement test oracles for non-testable programs. In Proceedings of the 26th international conference on software engineering and knowledge engineering (SEKE) (pp. 199–204). Knowledge Systems Institute: Vancouver.
Paleari, R, Martignoni, L, Roglia, GF, Bruschi, D. (2010). N-version disassembly: differential testing of x86 disassemblers. In Proceedings of the 19th international symposium on software testing and analysis (pp. 265–274). New York: ACM.
Patel, K, & Hierons, RM. (2015). The interlocutory tool box: techniques for curtailing coincidental correctness. Tech. rep., Brunel University.
Pezzè, M, & Zhang, C. (2014). Automated test oracles: a survey. In Memon, A (Ed.) Advances in computers (pp. 1–48): Elsevier.
Poon, PL, Kuo, FC, Liu, H, Chen, TY. (2014). How can non-technical end users effectively test their spreadsheets? Information Technology and People, 27(4), 440–462.
Article
Google Scholar
Popay, J, Roberts, H, Sowden, A, Petticrew, M, Arai, L, Rodgers, M, Britten N. (2006). Guidance on the conduct of narrative synthesis in systematic reviews: a product from the ESRC methods program. Lancaster University.
Segura, S, Hierons, RM, Benavides, D, Ruiz-Cortés, A. (2011). Automated metamorphic testing on the analyses of feature models. Information and Software Technology, 53(3), 245–258.
Article
Google Scholar
Ševčíková, H, Borning, A, Socha, D, Bleek, WG. (2006). Automated testing of stochastic systems: a statistically grounded approach. In Proceedings of the 2006 international symposium on software testing and analysis (pp. 215–224). New York: ACM.
Shepperd, M. (2013). Combining evidence and meta-analysis in software engineering. In Lucia AD, & Ferrucci, F (Eds.) Software engineering (pp. 46–70). Berlin: Springer.
Chapter
Google Scholar
Sim, KY, Wong, DML, Hii, TY. (2013). Evaluating the effectiveness of metamorphic testing on edge detection programs. International Journal of Innovation Management and Technology, 4(1), 6–10.
Google Scholar
Sim, KY, Low, CS, Kuo, FC. (2014). Eliminating human visual judgment from testing of financial charting software. Journal of Software, 9(2), 298–312.
Article
Google Scholar
Tiwari, S, Mishra, KK, Kumar, A, Misra, AK. (2011). Spectrum-based fault localization in regression testing. In Proceedings of the 8th international conference on information technology: new generations (ITNG) (pp. 191–195). Las Vegas: IEEE.
Weyuker, EJ. (1982). On testing non-testable programs. The Computer Journal, 25(4), 465–470.
Article
Google Scholar
Wu, P. (2005). In Proceedings of the 29th annual international computer software and applications conference (COMPSAC) (pp. 19–24). Edinburgh: IEEE.
Yoo, S. (2010). Metamorphic testing of stochastic optimisation. In Proceedings of the 3rd international conference on software testing, verification, and validation workshops (ICSTW) (pp. 192–201). Paris: IEEE.
Zhang, Z, Chan, WK, Tse, TH, Hu, P. (2009). Experimental study to compare the use of metamorphic testing and assertion checking. Journal of Software, 20(10), 2637–2654.
Article
Google Scholar
Arcaini, P, Gargantini, A, Riccobene, E. (2013). Combining model-based testing and runtime monitoring for program testing in the presence of nondeterminism. In Proceedings of the 6th international conference on software testing, verification and validation workshops (ICSTW) (pp. 178–187). Luxembourg: IEEE.
Asrafi, M, Liu, H, Kuo, FC. (2011). On testing effectiveness of metamorphic relations: a case study. In Proceedings of the 5th international conference on secure software integration and reliability improvement (SSIRI) (pp. 147–156). Jeju Island: IEEE.
Barr, ET, Harman, M, McMinn, P, Shahbaz, M, Yoo, S. (2015). The oracle problem in software testing: a survey. IEEE Transactions on Software Engineering, 41 (5), 507–525.
Article
Google Scholar
Barus, AC, Chen, TY, Grant, D, Kuo, FC, Lau, MF. (2011). Testing of heuristic methods: a case study of greedy algorithm. In Huzar Z, Koci, R, Meyer, B, Walter, B, Zendulka, J (Eds.) Software Engineering Techniques (pp. 246–260). Berlin: Springer.
Chapter
Google Scholar
Batra, G, & Sengupta, J. (2011). An efficient metamorphic testing technique using genetic algorithm. In Dua, S, Sahni, S, Goyal, DP (Eds.) Information intelligence, systems, technology and management (pp. 180–188). Berlin: Springer.
Chapter
Google Scholar
Chan, WK, & Tse, TH. (2013). Oracles are hardly attain’d, and hardly understood: confessions of software testing researchers. In Proceedings of the 13th international conference on quality software (QSIC) (pp. 245–252). Najing: IEEE.
Chan, FT, Chen, TY, Cheung, SC, Lau, MF, Yiu, SM. (1998). Application of metamorphic testing in numerical analysis. In Proceedings of the IASTED international conference on software engineering (pp. 191–197). Las Vegas: ACTA Press.
Chen, HY. (1999). The application of an algebraic design method to deal with oracle problem in object-oriented class level testing. In Proceedings of the international conference on systems, man, and cybernetics (SMC) (pp. 928–932). Tokyo: IEEE.
Chen, TY. (2010). Metamorphic testing: a simple approach to alleviate the oracle problem. In Proceedings of the 5th IEEE international symposium on service oriented system engineering (SOSE) (pp. 1–2). Nanjing: IEEE.
Chen, HY, Tse, TH, Chan, FT, Chen, TY. (1998a). In black and white: an integrated approach to class-level testing of object-oriented programs. ACM Transactions on Software Engineering and Methodology (TOSEM,) 7(3), 250–295.
Article
Google Scholar
Chen, HY, Tse, TH, Deng, YT. (2000). ROCS: an object-oriented class-level testing system based on the relevant observable contexts technique. Information and Software Technology, 42(10), 677–686.
Article
Google Scholar
Chen, TY, Feng, J, Tse, TH. (2002). Metamorphic testing of programs on partial differential equations: a case study. In Proceedings of the 26th annual international computer software and applications conference (COMPSAC) (pp. 327–333). Oxford: IEEE.
Chen, TY, Kuo, FC, Tse, TH, Zhou, ZQ. (2003a). Metamorphic testing and beyond. In Proceedings of the 11th annual international workshop on software technology and engineering practice (pp. 94–100). Amsterdam: IEEE.
Chen, TY, Kuo, FC, Liu, Y, Tang, A. (2004b). Metamorphic testing and testing with special values. In Proceedings of the 5th ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (pp. 128–134). Beijing: ACIS.
Chen, TY, Kuo, FC, Liu, H, Wang, S. (2009b). Conformance testing of network simulators based on metamorphic testing technique. In Proceedings of the joint 11th IFIP WG 6.1 international conference FMOODS and 29th IFIP WG 6.1 international conference FORTE (pp. 243–248). Portugal: Springer.
Chapter
Google Scholar
Chen, TY, Kuo, FC, Tam, WK, Merkel, R. (2011a). Testing a software-based PID controller using metamorphic testing, SciTePress - Science and Technology Publications, Algarve.
Chen, L, Cai, L, Liu, J, Liu, Z, Wei, S, Liu, P. (2012a). An optimized method for generating cases of metamorphic testing. In Proceedings of the 6th international conference on new trends in information science and service science and data mining (ISSDM) (pp. 439–443). Taipei: IEEE.
Chen, TY, Kuo, FC, Towey, D, Zhou, ZQ. (2012b). Metamorphic testing: applications and integration with other methods: tutorial synopsis. In Proceedings of the 12th international conference on quality software (QSIC) (pp. 285–288). Shaanxi: IEEE.
Ding, J, & Xu, D. (2012). Model-based metamorphic testing: a case study. In Proceedings of the 24th international conference on software engineering and knowledge engineering (SEKE) (pp. 363–368). San Francisco Bay: Knowledge Systems Institute Graduate School.
Ding, J, Wu, T, Lu, J, Hu, XH. (2010). Self-checked metamorphic testing of an image processing program. In Proceedings of the 4th international conference on secure software integration and reliability improvement (SSIRI) (pp. 190–197). Singapore: IEEE.
Ding, J, Wu, T, Wu, D, Lu, JQ, Hu, XH. (2011). Metamorphic testing of a monte carlo modeling program. In Proceedings of the 6th international workshop on automation of software test (pp. 1–7). New York: ACM.
Dong, G, Xu, B, Chen, L, Nie, C, Wang, L. (2008). Case studies on testing with compositional metamorphic relations. Journal of Southeast University, 24 (4), 437–443.
Google Scholar
Gore, R, Reynolds, PFJ, Kamensky, D. (2011). Statistical debugging with elastic predicates. In Proceedings of the 26th IEEE/ACM international conference on automated software engineering (ASE) (pp. 492–495). Kansas: IEEE.
Huang, S, Ji, MY, Hui, ZW, Duanmu, YT. (2011). Detecting integer bugs without oracle based on metamorphic testing technique. Applied Mechanics and Materials, 121–126, 1961–1965.
Article
Google Scholar
Hui, Z, Huang, S, Ren, Z, Yao, Y. (2013). Metamorphic testing integer overflow faults of mission critical program: a case study. Mathematical Problems in Engineering 1–6.
MathSciNet
Article
Google Scholar
Jiang, M, Chen, TY, Kuo, FC, Ding, Z. (2013). Testing central processing unit scheduling algorithms using metamorphic testing. In Proceedings of the 4th IEEE international conference on software engineering and service science (ICSESS) (pp. 530–536). Beijing: IEEE.
Just, R, & Schweiggert, F. (2011). Automating unit and integration testing with partial oracles. Software Quality Journal, 19(4), 753–769.
Article
Google Scholar
Kamensky, D, Gore, R, Reynolds, PFJ. (2011). Applying enhanced fault localization technology to Monte Carlo simulations. In Proceedings of the 2011 Winter Simulation Conference (WSC) (pp. 2798–2809). Arizona: IEEE.
Khosla, T, & Garg, S. (2011). Metamorphic testing effectiveness on trignometry. International Journal of Computer Science and Technology, 2(3), 576–578.
Google Scholar
Kim-Park, DS, Riva, C, Tuya, J. (2009). A partial test oracle for XML query testing. In Proceedings of the testing: academic and industrial conference on practice and research techniques (TAIC PART) (pp. 13–20). Windsor: IEEE.
Kim-Park, DS, Riva, C, Tuya, J. (2010). An automated test oracle for XML processing programs. In Proceedings of the 1st international workshop on software test output validation (pp. 5–12). New York: ACM.
Lei, Y, Mao, X, Chen, TY. (2013). Backward-slice-based statistical fault localization without test oracles. In Proceedings of the 13th international conference on quality software (QSIC) (pp. 212–221). Najing: IEEE.
Liu, H, Wang, D, Lin, H, Chen, TY. (2009). On the integration of metamorphic testing and model checking. In Proceedings of the IADIS international conference applied computing (pp. 299–302). Rome: IADIS Press.
Lu, XL, Dong, Yw, Luo, C. (2010). Testing of component-based software: a metamorphic testing methodology. In Proceedings of the 7th international conference on ubiquitous intelligence computing and 7th international conference on autonomic trusted computing (UIC/ATC) (pp. 272–276). Xian: IEEE.
Martignoni, L, Paleari, R, Roglia, GF, Bruschi, D. (2010). Testing system virtual machines. In Proceedings of the 19th international symposium on software testing and analysis (pp. 171–182). New York: ACM.
Nardo, DD, Alshahwan, N, Briand, LC, Fourneret, E, Nakic-Alfirevic, T, Masquelier, V. (2013). Model based test validation and oracles for data acquisition systems. In Proceedings of the 28th IEEE/ACM international conference on automated software engineering (ASE) (pp. 540–550). California: IEEE.
Oliveira, RAP, Delamaro, ME, Nunes, FLS. (2011). Exploring the OFIm framework to support the test of programs with GUIs. Tech. rep., University of Sao Paulo.
Padgham, L, Zhang, Z, Thangarajah, J, Miller, T. (2013). Model-based test oracle generation for automated unit testing of agent systems. IEEE Transactions on Software Engineering, 39(9), 1230–1244.
Article
Google Scholar
Poutakidis, D, Padgham, L, Winikoff, M. (2002). Debugging multi-agent systems using design artifacts: the case of interaction protocols. In Proceedings of the 1st international joint conference on autonomous agents and multiagent systems: part 2 (pp. 960–967). New York: ACM.
Poutakidis, D, Winikoff, M, Padgham, L, Zhang, Z. (2009). Debugging and testing of multi-agent systems using design artefacts. In Multi-agent programming (pp. 215–258). Boston: Springer.
Chapter
Google Scholar
Pullum, LL, & Ozmen, O. (2012). Early results from metamorphic testing of epidemiological models. In Proceedings of the ASE/IEEE international conference on biomedical computing (BioMedCom) (pp. 62–67). Washington: IEEE.
Rao, P, Zheng, Z, Chen, TY, Wang, N, Cai, K. (2013). Impacts of test suite’s class imbalance on spectrum-based fault localization techniques. In Proceedings of the 13th international conference on quality software (pp. 260–267). Najing: IEEE.
Roper, M. (2009). Artifical immune systems, danger theory, and the oracle problem. In Proceedings of the testing: academic and industrial conference on practice and research techniques (TAIC PART) (pp. 125–126). Windsor: IEEE.
Sadi, MS, Kuo, FC, Ho, JWK, Charleston, MA, Chen, TY. (2011). Verification of phylogenetic inference programs using metamorphic testing. Journal of Bioinformatics and Computational Biology, 9(6), 729–747.
Article
Google Scholar
Satpathy, M, Butler, M, Leuschel, M, Ramesh, S. (2007). Automatic testing from formal specifications. In Tests and proofs (pp. 95–113): Springer.
Segura, S, Durán, A, Sánchez, AB, Le Berre, D, Lonca, E, Ruiz-Cortés, A. (2013). Automated metamorphic testing on the analysis of software variability. Tech. Rep. SA-2013-TR-03, University of Seville.
Sim, KY, Pao, WKS, Lin, C. (2005). Metamorphic testing using geometric interrogation technique and its application. In Proceedings of the 2nd international conference of electrical engineering/electronics, computer, telecommunications, and information technology (ECTI) (pp. 91–95).
Sim, KY, Low, CS, Kuo, FC. (2010). Detecting faults in technical indicator computations for financial market analysis. In Proceedings of the 2nd international conference on information science and engineering (ICISE) (pp. 2749–2754). Hangzhou: IEEE.
Singh, G, & Singh, G. (2012). An automated metamorphic testing technique for designing effective metamorphic relations. In Parashar M, Kaushik, D, Rana, OF, Samtaney, R, Yang, Y, Zomaya, A (Eds.) Contemporary computing (pp. 152–163). Berlin: Springer.
Sánchez, AB, & Segura, S. (2012). Automated testing on the analysis of variability-intensive artifacts: an exploratory study with SAT solvers. Tech. rep., University of Seville.
Tao, Q, Wu, W, Zhao, C, Shen, W. (2010). An automatic testing approach for compiler based on metamorphic testing technique. In Proceedings of the 17th Asia Pacific software engineering conference (APSEC) (pp. 270–279). Sydney: IEEE.
Tse, TH. (2005). Research directions on model-based metamorphic testing and verification. In Proceedings of the 29th annual international computer software and applications conference (COMPSAC) (p. 1). Edinburgh: IEEE.
Tse, TH, Chen, TY, Zhou, Z. (2000). Testing of large number multiplication functions in cryptographic systems. In Proceedings of the 1st Asia-Pacific conference on quality software (pp. 89–98). Hong Kong: IEEE.
Tse, TH, Lau, FCM, Chan, WK, Liu, PCK, Luk, CKF. (2007). Testing object-oriented industrial software without precise oracles or results. Communications of the ACM, 50(8), 78–85.
Article
Google Scholar
Weyuker, EJ. (1980). The oracle assumption of program testing. In Proceedings of the 13th international conference on system sciences (ICSS) (pp. 44–49). Hawaii: Western Periodicals.
Wu, P, Shi, Xc, Tang, Jj, Lin, Hm, Chen, TY. (2005). Metamorphic testing and special case testing: a case study. Journal of Software, 16(7), 1210–1220.
Article
Google Scholar
Xie, X, Ho, JWK, Murphy, C, Kaiser, G, Xu, B, Chen, TY. (2011). Testing and validating machine learning classifiers by metamorphic testing. Journal of Systems and Software, 84(4), 544–558.
Article
Google Scholar
Xie, X, Wong, WE, Chen, TY, Xu, B. (2013). Metamorphic slice: an application in spectrum-based fault localization. Information and Software Technology, 55(5), 866–879.
Article
Google Scholar
Yao, Y, Huang, S, Ji, M. (2012). Research on metamorphic testing for oracle problem of integer bugs. In Jin D, & Lin, S (Eds.) Advances in computer science and information engineering (pp. 95–100). Berlin: Springer.
Google Scholar
Yi, Y, Changyou, Z, Song, H, Zhengping, R. (2013). Research on metamorphic testing: a case study in integer bugs detection. In Proceedings of the 4th international conference on intelligent systems design and engineering applications (pp. 488–493). China: IEEE.
Zhang, J, Hu, X, Zhang, B. (2011). An evaluation approach for the program of association rules algorithm based on metamorphic relations. Journal of Electronics (China), 28(4-6), 623–631.
Article
Google Scholar
Zheng, W, Ma, H, Lyu, MR, Xie, T, King, I. (2011). Mining test oracles of web search engines. In Proceedings of the 26th IEEE/ACM international conference on automated software engineering (ASE) (pp. 408–411). Kansas: IEEE.
Zhou, ZQ, Huang, DH, Tse, TH, Yang, Z, Huang, H, Chen, TY. (2004). Metamorphic testing and its applications. In Proceedings of the 8th international symposium on future software technology (ISFST) (pp. 1–6). The Software Engineers Association: Xian.
Zhou, ZQ, Tse, TH, Kuo, FC, Chen, TY. (2007). Automated functional testing of web search engines in the absence of an oracle. Tech. Rep. TR-2007-06, The University of Hong Kong.