Skip to main content
SpringerLink
Log in

EdSketch: execution-driven sketching for Java

  • SPIN 2017
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Sketching is a synthesis approach that allows users to provide high-level insights into a synthesis problem and let synthesis tools complete low-level details. Users write sketches—partial programs that have “holes” and provide test assertions as the correctness criteria. The sketching techniques fill the holes with code fragments such that the complete program satisfies all test assertions. Traditional techniques translate the sketching problem to propositional satisfiability formulas and leverage SAT solvers to generate programs with the desired functionality. While effective for a range of small well-defined domains, such translation-based approaches have a key limitation when applying to real applications: They require either translating all relevant libraries that are invoked directly or indirectly by the given sketch or creating models of those libraries, which requires much manual effort. This paper introduces execution-driven sketching, a novel approach for synthesizing Java programs with on-demand candidate generation. The key novelty of our work is to leverage runtime behavior to prune a large amount of search space. EdSketch explores the actual program behaviors in the presence of libraries and sketches small parts of real-world applications, which may use complex constructs of modern languages, such as reflection, native calls and File I/O. We further leverage a set of pruning strategies based on Java syntax to expedite the synthesis process. EdSketch embodies our approach in two forms: a stateful search based on the Java PathFinder model checker; and a stateless search based on re-execution inspired by the VeriSoft model checker. Experimental results show that EdSketch can complete some sketches that contain complex constructs in the presence of libraries, recursive procedures and advanced features like reflection. Without translating to SAT, EdSketch ’s performance compares well with the SAT-based Sketch system for a range of small but complex data structure subjects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Solar-Lezama, A.: Program sketching. STTT 15(5–6), 475–495 (2013)

    Article  Google Scholar 

  2. Srivastava, S., Gulwani, S., Foster, J.S.: From program verification to program synthesis. In: Proceedings of the 37th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2010, Madrid, Spain, January 17–23, 2010, pp. 313–326 (2010)

  3. Kuncak, V., Mayer, M., Piskac, R., Suter, P.: Complete functional synthesis. In: Proceedings of the 2010 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2010, Toronto, Ontario, Canada, June 5–10, 2010, pp. 316–329 (2010)

  4. Alur, R., Bodík, R., Juniwal, G., Martin, M.M.K., Raghothaman, M., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. In: Formal Methods in Computer-Aided Design, FMCAD 2013, Portland, OR, USA, 20–23 October 2013, pp. 1–8 (2013)

  5. Feser, J.K., Chaudhuri, S., Dillig, I.: Synthesizing data structure transformations from input–output examples. In: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, June 15–17, 2015, pp. 229–239 (2015)

  6. Jeon, J., Qiu, X., Foster, J.S., Solar-Lezama, A.: JSketch: sketching for Java. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, Bergamo, Italy, August 30–September 4, 2015, pp. 934–937 (2015)

  7. Visser, W., Havelund, K., Brat, G.P., Park, S.: Model checking programs. In: ASE 2000, pp. 3–12 (2000)

  8. Godefroid, P.: Model checking for programming languages using verisoft. In: POPL 1997, pp. 174–186 (1997)

  9. Solar-Lezama, A., Tancau, L., Bodík, R., Seshia, S.A., Saraswat, V.A.: Combinatorial sketching for finite programs. In: Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2006, San Jose, CA, USA, October 21–25, 2006, pp. 404–415 (2006)

  10. http://docs.oracle.com/javase/7/docs/api/java/lang/Class.html (2017). Accessed 30 Jan 2017

  11. Ujma, M., Shafiei, N.: jpf-concurrent: An extension of java pathfinder for java.util.concurrent. CoRR arXiv:1205.0042 (2012)

  12. Elkarablieh, B., Khurshid, S.: Juzi: a tool for repairing complex data structures. In: ICSE 2008, pp. 855–858 (2008)

  13. Albarghouthi, A., Gulwani, S., Kincaid, Z.: Recursive program synthesis. In: Computer Aided Verification—25th International Conference, CAV 2013, Saint Petersburg, Russia, 13–19 July 2013. Proceedings, pp. 934–950 (2013)

  14. Inala, J.P., Polikarpova, N., Qiu, X., Lerner, B.S., Solar-Lezama, A.: Synthesis of recursive ADT transformations from reusable templates. In: Tools and Algorithms for the Construction and Analysis of Systems—23rd International Conference, TACAS 2017, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2017, Uppsala, Sweden, April 22–29, 2017, Proceedings, Part I, pp. 247–263 (2017)

  15. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn. MIT Press, Cambridge (2009)

    MATH  Google Scholar 

  16. https://github.com/SketchFix/EdSketch-Evaluation (2019). Accessed 1 Jan 2019

  17. Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: ASE 2005, pp. 273–282 (2005)

  18. Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on Java predicates. ISSTA 2002, 123–133 (2002)

    Article  Google Scholar 

  19. Bodík, R., Chandra, S., Galenson, J., Kimelman, D., Tung, N., Barman, S., Rodarmor, C.: Programming with angelic nondeterminism. In: Proceedings of the 37th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2010, Madrid, Spain, 17–23 January 2010, pp. 339–352 (2010)

  20. Galenson, J., Reames, P., Bodík, R., Hartmann, B., Sen, K.: Codehint: dynamic and interactive synthesis of code snippets. In: 36th International Conference on Software Engineering, ICSE ’14, Hyderabad, India, May 31–June 07, 2014, pp. 653–663 (2014)

  21. Feng, Y., Martins, R., Wang, Y., Dillig, I., Reps, T.W.: Component-based synthesis for complex APIs. In: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, January 18–20, 2017, pp. 599–612 (2017)

  22. Yang, Z., Hua, J., Wang, K., Khurshid, S.: EdSynth: Synthesizing API sequences with conditionals and loops. In: 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2018, Vasteras, Sweden, April 9–13, 2018. IEEE Computer Society (2018)

  23. Kneuss, E., Kuraj, I., Kuncak, V., Suter, P.: Synthesis modulo recursive functions. In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, part of SPLASH 2013, Indianapolis, IN, USA, October 26–31, 2013, pp. 407–426 (2013)

  24. https://sourceforge.net/p/jfreechart/code/HEAD/tree/ (2018). Accessed 30 April 2018

  25. https://github.com/google/closure-library (2018). Accessed 30 April 2018

  26. https://github.com/apache/commons-lang (2018). Accessed 30 April 2018

  27. https://github.com/apache/commons-math (2018). Accessed 30 April 2018

  28. https://github.com/JodaOrg/joda-time (2018). Accessed 30 April 2018

  29. Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are mutants a valid substitute for real faults in software testing? In: FSE 2014, pp. 654–665 (2014)

  30. Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for java programs. In: ISSTA ’14, San Jose, CA, USA, July 21–26, 2014, pp. 437–440 (2014)

  31. Hua, J., Zhang, M., Wang, K., Khurshid, S.: Towards practical program repair with on-demand candidate generation. In: Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27–June 3, 2018. ACM (2018)

  32. Jha, S., Gulwani, S., Seshia, S.A., Tiwari, A.: Oracle-guided component-based program synthesis. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering—Volume 1, ICSE 2010, Cape Town, South Africa, 1–8 May 2010, pp. 215–224 (2010)

  33. Singh, R., Solar-Lezama, A.: Synthesizing data structure manipulations from storyboards. In: SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13th European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5–9, 2011, pp. 289–299 (2011)

  34. Gvero, T., Kuncak, V., Piskac, R.: Interactive synthesis of code snippets. In: Computer Aided Verification—23rd International Conference, CAV 2011, Snowbird, UT, USA, July 14-20, 2011. Proceedings, pp. 418–423 (2011)

  35. Wang, K., Sullivan, A., Marinov, D., Khurshid, S.: ASketch: a sketching framework for alloy. In: Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2018, pp. 916–919 (2018)

  36. Jackson, D.: Software Abstractions: Logic, Language, and Analysis. The MIT Press, Cambridge (2006)

    Google Scholar 

  37. Mandelin, D., Xu, L., Bodík, R., Kimelman, D.: Jungloid mining: helping to navigate the API jungle. In: Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation, Chicago, IL, USA, June 12–15, 2005, pp. 48–61 (2005)

  38. Floyd, R.W.: Nondeterministic algorithms. J. ACM 14, 4 (1967)

    Article  MATH  Google Scholar 

  39. Perelman, D., Gulwani, S., Ball, T., Grossman, D.: Type-directed completion of partial expressions. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’12, Beijing, China, June 11–16, 2012, pp. 275–286 (2012)

  40. Raychev, V., Vechev, M.T., Yahav, E.: Code completion with statistical language models. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom, June 09–11, 2014, pp. 419–428 (2014)

  41. Holmes, R., Murphy, G.C.: Using structural context to recommend source code examples. In: 27th International Conference on Software Engineering (ICSE 2005), 15–21 May 2005, St. Louis, Missouri, USA, pp. 117–125 (2005)

  42. Malik, M.Z., Ghori, K., Elkarablieh, B., Khurshid, S.: A case for automated debugging using data structure repair. In: ASE, pp. 620–624 (2009)

  43. Gopinath, D., Malik, M.Z., Khurshid, S.: Specification-based program repair using SAT. In: TACAS 2011, pp. 173–188 (2011)

  44. Weimer, W., Fry, Z.P., Forrest, S.: Leveraging program equivalence for adaptive program repair: models and first results. In: ASE, pp. 356–366 (2013)

  45. Long, F., Rinard, M.: Staged program repair with condition synthesis. In: ESEC/FSE, pp. 166–178 (2015)

  46. Mechtaev, S., Yi, J., Roychoudhury, A.: Angelix: Scalable multiline program patch synthesis via symbolic analysis. In: ICSE 2016 (2016)

  47. Hua, J., Khurshid, S.: A sketching-based approach for debugging using test cases. In: ATVA 2016, pp. 463–478 (2016)

Download references

Acknowledgements

This work was funded in part by the National Science Foundation (NSF Grant Nos. CCF-1319688 and CNS-1239498), Shenzhen Peacock Plan (Grant No. KQT D2016112514355531), the Science and Technology Innovation Committee Foundation of Shenzhen (Grant No. JCYJ2017081 7110848086). We thank Mukul Prasad, Allison Sullivan and Kaiyuan Wang for discussion and comments.

Author information

Authors and Affiliations

  1. The University of Texas at Austin, Austin, TX, USA

    Jinru Hua & Sarfraz Khurshid

  2. Southern University of Science and Technology, Shenzhen, China

    Yushan Zhang & Yuqun Zhang

Authors
  1. Jinru Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yushan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuqun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sarfraz Khurshid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuqun Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hua, J., Zhang, Y., Zhang, Y. et al. EdSketch: execution-driven sketching for Java. Int J Softw Tools Technol Transfer 21, 249–265 (2019). https://doi.org/10.1007/s10009-019-00512-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-019-00512-8

Keywords

Search

Navigation