Abstract
Program assertions are useful for many program analysis tasks. They are however often missing in practice. Many approaches have been developed to generate assertions automatically. Existing methods are either based on generalizing from a set of test cases (e.g., Daikon), or based on some forms of symbolic execution. In this work, we develop a novel approach for generating likely assertions automatically based on active learning. Our targets are complex Java programs which are challenging for symbolic execution. Our key idea is to generate candidate assertions based on test cases and then apply active learning techniques to iteratively improve them. We evaluate our approach using two sets of programs, i.e., 425 methods from three popular Java projects from GitHub and 10 programs from the SVComp repository. We evaluate the ‘correctness’ of the assertions either by comparing them with existing assertion-like checking conditions, or by comparing them with the documentation, or by verifying them.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Refer to recent development on supporting polymorphism in symbolic execution in [17].
References
Alur, R., Černỳ, P., Madhusudan, P., Nam, W.: Synthesis of interface specifications for java classes. In: POPL, pp. 98–109. ACM (2005)
Beller, M., Gousios, G., Panichella, A., Zaidman, A.: When, how, and why developers (do not) test in their IDEs. In: ESEC/FSE, pp. 179–190. ACM (2015)
Beller, M., Gousios, G., Zaidman, A.: How (much) do developers test? In: ICSE, pp. 559–562. IEEE (2015)
Boshernitsan, M., Doong, R., Savoia, A.: From daikon to agitator: lessons and challenges in building a commercial tool for developer testing. In: ISSTA, pp. 169–180. ACM (2006)
Bshouty, N.H., Goldman, S.A., Mathias, H.D., Suri, S., Tamaki, H.: Noise-tolerant distribution-free learning of general geometric concepts. JACM 45(5), 863–890 (1998)
Chapelle, O.: Training a support vector machine in the primal. Neural Comput. 19(5), 1155–1178 (2007)
Cohn, D.: Active learning. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 10–14. Springer, New York (2010)
Csallner, C., Tillmann, N., Smaragdakis, Y.: DySy: Dynamic symbolic execution for invariant inference. In: ICSE, pp. 281–290. ACM (2008)
Ernst, M.D., Cockrell, J., Griswold, W.G., Notkin, D.: Dynamically discovering likely program invariants to support program evolution. IEEE Trans. Software Eng. 27(2), 99–123 (2001)
Ernst, M.D., Perkins, J.H., Guo, P.J., McCamant, S., Pacheco, C., Tschantz, M.S., Xiao, C.: The daikon system for dynamic detection of likely invariants. Sci. Comput. Program. 69(1), 35–45 (2007)
Garg, P., Löding, C., Madhusudan, P., Neider, D.: ICE: a robust framework for learning invariants. In: Biere, A., Bloem, R. (eds.) CAV 2014. LNCS, vol. 8559, pp. 69–87. Springer, Cham (2014). doi:10.1007/978-3-319-08867-9_5
Garg, P., Neider, D., Madhusudan, P., Roth, D.: Learning invariants using decision trees and implication counterexamples. In: POPL, pp. 499–512. ACM (2016)
Hoare, C.A.R.: Assertions: a personal perspective. IEEE Ann. Hist. Comput. 25(2), 14–25 (2003)
Krishna, S., Puhrsch, C., Wies, T.: Learning invariants using decision trees. arXiv preprint arXiv:1501.04725 (2015)
Li, L., Lu, Y., Xue, J.: Dynamic symbolic execution for polymorphism. In: CC, pp. 120–130. ACM (2017)
Nguyen, T., Kapur, D., Weimer, W., Forrest, S.: DIG: a dynamic invariant generator for polynomial and array invariants. ACM Trans. Softw. Eng. Methodol. 23(4), 30 (2014)
Nguyen, T., Kapur, D., Weimer, W., Forrest, S.: Using dynamic analysis to generate disjunctive invariants. In: ICSE, pp. 608–619. ACM (2014)
Pacheco, C., Lahiri, S.K., Ernst, M.D., Ball, T.: Feedback-directed random test generation. In: ICSE, pp. 75–84. IEEE (2007)
Padhi, S., Sharma, R., Millstein, T.: Data-driven precondition inference with learned features. In: PLDI, pp. 42–56. ACM (2016)
Sankaranarayanan, S., Chaudhuri, S., Ivančić, F., Gupta, A.: Dynamic inference of likely data preconditions over predicates by tree learning. In: ISSTA, pp. 295–306. ACM (2008)
Schohn, G., Cohn, D.: Less is more: active learning with support vector machines. In: ICML, pp. 839–846 (2000)
Sharma, R., Aiken, A.: From invariant checking to invariant inference using randomized search. In: Biere, A., Bloem, R. (eds.) CAV 2014. LNCS, vol. 8559, pp. 88–105. Springer, Cham (2014). doi:10.1007/978-3-319-08867-9_6
Sharma, R., Gupta, S., Hariharan, B., Aiken, A., Liang, P., Nori, A.V.: A data driven approach for algebraic loop invariants. In: Felleisen, M., Gardner, P. (eds.) ESOP 2013. LNCS, vol. 7792, pp. 574–592. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37036-6_31
Sharma, R., Gupta, S., Hariharan, B., Aiken, A., Nori, A.V.: Verification as learning geometric concepts. In: Logozzo, F., Fähndrich, M. (eds.) SAS 2013. LNCS, vol. 7935, pp. 388–411. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38856-9_21
Sharma, R., Nori, A.V., Aiken, A.: Interpolants as classifiers. In: Madhusudan, P., Seshia, S.A. (eds.) CAV 2012. LNCS, vol. 7358, pp. 71–87. Springer, Heidelberg (2012). doi:10.1007/978-3-642-31424-7_11
Somenzi, F., Bradley, A.R.: IC3: where monolithic and incremental meet. In: FMCAD, pp. 3–8 (2011)
Sun, J., Xiao, H., Liu, Y., Lin, S., Qin, S.: TLV: abstraction through testing, learning, and validation. In: ESEC/FSE, pp. 698–709. ACM (2015)
Tong, S., Chang, E.Y.: Support vector machine active learning for image retrieval. In: MULTIMEDIA, pp. 107–118. ACM(2001)
Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. J. Mach. Learn. Res. 2, 45–66 (2001)
Wei, Y., Furia, C.A., Kazmin, N., Meyer, B.: Inferring better contracts. In: ICSE, pp. 191–200. ACM (2011)
Xiao, H., Sun, J., Liu, Y., Lin, S., Sun, C.: TzuYu: Learning stateful typestates. In: ASE, pp. 432–442. IEEE (2013)
Xie, T., Notkin, D.: Mutually enhancing test generation and specification inference. In: Petrenko, A., Ulrich, A. (eds.) FATES 2003. LNCS, vol. 2931, pp. 60–69. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24617-6_5
Zhang, L., Yang, G., Rungta, N., Person, S., Khurshid, S.: Feedback-driven dynamic invariant discovery. In: ISSTA, pp. 362–372. ACM (2014)
Acknowledgments
This research was funded by the project T2MOE1704.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
H. Pham, L., Tran Thi, L.L., Sun, J. (2017). Assertion Generation Through Active Learning. In: Duan, Z., Ong, L. (eds) Formal Methods and Software Engineering. ICFEM 2017. Lecture Notes in Computer Science(), vol 10610. Springer, Cham. https://doi.org/10.1007/978-3-319-68690-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-68690-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68689-9
Online ISBN: 978-3-319-68690-5
eBook Packages: Computer ScienceComputer Science (R0)