EdSketch: execution-driven sketching for Java

  • Jinru HuaEmail author
  • Yushan Zhang
  • Yuqun Zhang
  • Sarfraz Khurshid
SPIN 2017


Sketching is a synthesis approach that allows users to provide high-level insights into a synthesis problem and let synthesis tools complete low-level details. Users write sketches—partial programs that have “holes” and provide test assertions as the correctness criteria. The sketching techniques fill the holes with code fragments such that the complete program satisfies all test assertions. Traditional techniques translate the sketching problem to propositional satisfiability formulas and leverage SAT solvers to generate programs with the desired functionality. While effective for a range of small well-defined domains, such translation-based approaches have a key limitation when applying to real applications: They require either translating all relevant libraries that are invoked directly or indirectly by the given sketch or creating models of those libraries, which requires much manual effort. This paper introduces execution-driven sketching, a novel approach for synthesizing Java programs with on-demand candidate generation. The key novelty of our work is to leverage runtime behavior to prune a large amount of search space. EdSketch explores the actual program behaviors in the presence of libraries and sketches small parts of real-world applications, which may use complex constructs of modern languages, such as reflection, native calls and File I/O. We further leverage a set of pruning strategies based on Java syntax to expedite the synthesis process. EdSketch embodies our approach in two forms: a stateful search based on the Java PathFinder model checker; and a stateless search based on re-execution inspired by the VeriSoft model checker. Experimental results show that EdSketch can complete some sketches that contain complex constructs in the presence of libraries, recursive procedures and advanced features like reflection. Without translating to SAT, EdSketch ’s performance compares well with the SAT-based Sketch system for a range of small but complex data structure subjects.


Program sketching Execution-driven synthesis Backtracking search 



This work was funded in part by the National Science Foundation (NSF Grant Nos. CCF-1319688 and CNS-1239498), Shenzhen Peacock Plan (Grant No. KQT D2016112514355531), the Science and Technology Innovation Committee Foundation of Shenzhen (Grant No. JCYJ2017081 7110848086). We thank Mukul Prasad, Allison Sullivan and Kaiyuan Wang for discussion and comments.


  1. 1.
    Solar-Lezama, A.: Program sketching. STTT 15(5–6), 475–495 (2013)CrossRefGoogle Scholar
  2. 2.
    Srivastava, S., Gulwani, S., Foster, J.S.: From program verification to program synthesis. In: Proceedings of the 37th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2010, Madrid, Spain, January 17–23, 2010, pp. 313–326 (2010)Google Scholar
  3. 3.
    Kuncak, V., Mayer, M., Piskac, R., Suter, P.: Complete functional synthesis. In: Proceedings of the 2010 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2010, Toronto, Ontario, Canada, June 5–10, 2010, pp. 316–329 (2010)Google Scholar
  4. 4.
    Alur, R., Bodík, R., Juniwal, G., Martin, M.M.K., Raghothaman, M., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. In: Formal Methods in Computer-Aided Design, FMCAD 2013, Portland, OR, USA, 20–23 October 2013, pp. 1–8 (2013)Google Scholar
  5. 5.
    Feser, J.K., Chaudhuri, S., Dillig, I.: Synthesizing data structure transformations from input–output examples. In: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, June 15–17, 2015, pp. 229–239 (2015)Google Scholar
  6. 6.
    Jeon, J., Qiu, X., Foster, J.S., Solar-Lezama, A.: JSketch: sketching for Java. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, Bergamo, Italy, August 30–September 4, 2015, pp. 934–937 (2015)Google Scholar
  7. 7.
    Visser, W., Havelund, K., Brat, G.P., Park, S.: Model checking programs. In: ASE 2000, pp. 3–12 (2000)Google Scholar
  8. 8.
    Godefroid, P.: Model checking for programming languages using verisoft. In: POPL 1997, pp. 174–186 (1997)Google Scholar
  9. 9.
    Solar-Lezama, A., Tancau, L., Bodík, R., Seshia, S.A., Saraswat, V.A.: Combinatorial sketching for finite programs. In: Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2006, San Jose, CA, USA, October 21–25, 2006, pp. 404–415 (2006)Google Scholar
  10. 10.
  11. 11.
    Ujma, M., Shafiei, N.: jpf-concurrent: An extension of java pathfinder for java.util.concurrent. CoRR arXiv:1205.0042 (2012)
  12. 12.
    Elkarablieh, B., Khurshid, S.: Juzi: a tool for repairing complex data structures. In: ICSE 2008, pp. 855–858 (2008)Google Scholar
  13. 13.
    Albarghouthi, A., Gulwani, S., Kincaid, Z.: Recursive program synthesis. In: Computer Aided Verification—25th International Conference, CAV 2013, Saint Petersburg, Russia, 13–19 July 2013. Proceedings, pp. 934–950 (2013)Google Scholar
  14. 14.
    Inala, J.P., Polikarpova, N., Qiu, X., Lerner, B.S., Solar-Lezama, A.: Synthesis of recursive ADT transformations from reusable templates. In: Tools and Algorithms for the Construction and Analysis of Systems—23rd International Conference, TACAS 2017, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2017, Uppsala, Sweden, April 22–29, 2017, Proceedings, Part I, pp. 247–263 (2017)Google Scholar
  15. 15.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn. MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  16. 16.
  17. 17.
    Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: ASE 2005, pp. 273–282 (2005)Google Scholar
  18. 18.
    Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on Java predicates. ISSTA 2002, 123–133 (2002)CrossRefGoogle Scholar
  19. 19.
    Bodík, R., Chandra, S., Galenson, J., Kimelman, D., Tung, N., Barman, S., Rodarmor, C.: Programming with angelic nondeterminism. In: Proceedings of the 37th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2010, Madrid, Spain, 17–23 January 2010, pp. 339–352 (2010)Google Scholar
  20. 20.
    Galenson, J., Reames, P., Bodík, R., Hartmann, B., Sen, K.: Codehint: dynamic and interactive synthesis of code snippets. In: 36th International Conference on Software Engineering, ICSE ’14, Hyderabad, India, May 31–June 07, 2014, pp. 653–663 (2014)Google Scholar
  21. 21.
    Feng, Y., Martins, R., Wang, Y., Dillig, I., Reps, T.W.: Component-based synthesis for complex APIs. In: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, January 18–20, 2017, pp. 599–612 (2017)Google Scholar
  22. 22.
    Yang, Z., Hua, J., Wang, K., Khurshid, S.: EdSynth: Synthesizing API sequences with conditionals and loops. In: 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2018, Vasteras, Sweden, April 9–13, 2018. IEEE Computer Society (2018)Google Scholar
  23. 23.
    Kneuss, E., Kuraj, I., Kuncak, V., Suter, P.: Synthesis modulo recursive functions. In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, part of SPLASH 2013, Indianapolis, IN, USA, October 26–31, 2013, pp. 407–426 (2013)Google Scholar
  24. 24.
  25. 25. (2018). Accessed 30 April 2018
  26. 26. (2018). Accessed 30 April 2018
  27. 27. (2018). Accessed 30 April 2018
  28. 28. (2018). Accessed 30 April 2018
  29. 29.
    Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are mutants a valid substitute for real faults in software testing? In: FSE 2014, pp. 654–665 (2014)Google Scholar
  30. 30.
    Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for java programs. In: ISSTA ’14, San Jose, CA, USA, July 21–26, 2014, pp. 437–440 (2014)Google Scholar
  31. 31.
    Hua, J., Zhang, M., Wang, K., Khurshid, S.: Towards practical program repair with on-demand candidate generation. In: Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27–June 3, 2018. ACM (2018)Google Scholar
  32. 32.
    Jha, S., Gulwani, S., Seshia, S.A., Tiwari, A.: Oracle-guided component-based program synthesis. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering—Volume 1, ICSE 2010, Cape Town, South Africa, 1–8 May 2010, pp. 215–224 (2010)Google Scholar
  33. 33.
    Singh, R., Solar-Lezama, A.: Synthesizing data structure manipulations from storyboards. In: SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13th European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5–9, 2011, pp. 289–299 (2011)Google Scholar
  34. 34.
    Gvero, T., Kuncak, V., Piskac, R.: Interactive synthesis of code snippets. In: Computer Aided Verification—23rd International Conference, CAV 2011, Snowbird, UT, USA, July 14-20, 2011. Proceedings, pp. 418–423 (2011)Google Scholar
  35. 35.
    Wang, K., Sullivan, A., Marinov, D., Khurshid, S.: ASketch: a sketching framework for alloy. In: Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2018, pp. 916–919 (2018)Google Scholar
  36. 36.
    Jackson, D.: Software Abstractions: Logic, Language, and Analysis. The MIT Press, Cambridge (2006)Google Scholar
  37. 37.
    Mandelin, D., Xu, L., Bodík, R., Kimelman, D.: Jungloid mining: helping to navigate the API jungle. In: Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation, Chicago, IL, USA, June 12–15, 2005, pp. 48–61 (2005)Google Scholar
  38. 38.
    Floyd, R.W.: Nondeterministic algorithms. J. ACM 14, 4 (1967)CrossRefzbMATHGoogle Scholar
  39. 39.
    Perelman, D., Gulwani, S., Ball, T., Grossman, D.: Type-directed completion of partial expressions. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’12, Beijing, China, June 11–16, 2012, pp. 275–286 (2012)Google Scholar
  40. 40.
    Raychev, V., Vechev, M.T., Yahav, E.: Code completion with statistical language models. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom, June 09–11, 2014, pp. 419–428 (2014)Google Scholar
  41. 41.
    Holmes, R., Murphy, G.C.: Using structural context to recommend source code examples. In: 27th International Conference on Software Engineering (ICSE 2005), 15–21 May 2005, St. Louis, Missouri, USA, pp. 117–125 (2005)Google Scholar
  42. 42.
    Malik, M.Z., Ghori, K., Elkarablieh, B., Khurshid, S.: A case for automated debugging using data structure repair. In: ASE, pp. 620–624 (2009)Google Scholar
  43. 43.
    Gopinath, D., Malik, M.Z., Khurshid, S.: Specification-based program repair using SAT. In: TACAS 2011, pp. 173–188 (2011)Google Scholar
  44. 44.
    Weimer, W., Fry, Z.P., Forrest, S.: Leveraging program equivalence for adaptive program repair: models and first results. In: ASE, pp. 356–366 (2013)Google Scholar
  45. 45.
    Long, F., Rinard, M.: Staged program repair with condition synthesis. In: ESEC/FSE, pp. 166–178 (2015)Google Scholar
  46. 46.
    Mechtaev, S., Yi, J., Roychoudhury, A.: Angelix: Scalable multiline program patch synthesis via symbolic analysis. In: ICSE 2016 (2016)Google Scholar
  47. 47.
    Hua, J., Khurshid, S.: A sketching-based approach for debugging using test cases. In: ATVA 2016, pp. 463–478 (2016)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Jinru Hua
    • 1
    Email author
  • Yushan Zhang
    • 2
  • Yuqun Zhang
    • 2
  • Sarfraz Khurshid
    • 1
  1. 1.The University of Texas at AustinAustinUSA
  2. 2.Southern University of Science and TechnologyShenzhenChina

Personalised recommendations