Abstract
Programming by Examples (PBE) involves synthesizing intended programs in an underlying domain-specific language from example-based specifications. PBE systems are already revolutionizing the application domain of data wrangling and are set to significantly impact several other domains including code refactoring.
There are three key components in a PBE system. (i) A search algorithm that can efficiently search for programs that are consistent with the examples provided by the user. We leverage a divide-and-conquer-based deductive search paradigm that inductively reduces the problem of synthesizing a program expression of a certain kind that satisfies a given specification into sub-problems that refer to sub-expressions or sub-specifications. (ii) Program ranking techniques to pick an intended program from among the many that satisfy the examples provided by the user. We leverage features of the program structure as well of the outputs generated by the program on test inputs. (iii) User interaction models to facilitate usability and debuggability. We leverage active-learning techniques based on clustering inputs and synthesizing multiple programs.
Each of these PBE components leverage both symbolic reasoning and heuristics. We make the case for synthesizing these heuristics from training data using appropriate machine learning methods. This can not only lead to better heuristics, but can also enable easier development, maintenance, and even personalization of a PBE system.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gulwani, S., Polozov, O., Singh, R.: Program synthesis. Found. Trends Program. Lang. 4(1–2), 1–119 (2017)
Gulwani, S.: Programming by examples - and its applications in data wrangling. In: Dependable Software Systems Engineering, pp. 137–158 (2016)
Le, V., Gulwani, S.: FlashExtract: a framework for data extraction by examples. In: PLDI, pp. 542–553 (2014)
Gulwani, S.: Automating string processing in spreadsheets using input-output examples. In: POPL, pp. 317–330 (2011)
Singh, R., Gulwani, S.: Learning semantic string transformations from examples. PVLDB 5(8), 740–751 (2012)
Singh, R., Gulwani, S.: Synthesizing number transformations from input-output examples. In: CAV, pp. 634–651 (2012)
Singh, R., Gulwani, S.: Transforming spreadsheet data types using examples. In: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20–22, 2016, pp. 343–356 (2016)
Barowy, D.W., Gulwani, S., Hart, T., Zorn, B.G.: FlashRelate: extracting relational data from semi-structured spreadsheets using examples. In: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, June 15–17, 2015, pp. 218–228 (2015)
Harris, W.R., Gulwani, S.: Spreadsheet table transformations from examples. In: PLDI, pp. 317–328 (2011)
Raza, M., Gulwani, S., Milic-Frayling, N.: Programming by example using least general generalizations. In: AAAI, pp. 283–290 (2014)
Rolim, R., Soares, G., D’Antoni, L., Polozov, O., Gulwani, S., Gheyi, R., Suzuki, R., Hartmann, B.: Learning syntactic program transformations from examples. In: Proceedings of the 39th International Conference on Software Engineering, ICSE 2017, Buenos Aires, Argentina, May 20–28, 2017, pp. 404–415 (2017)
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwinska, A., Colmenarejo, S.G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A.P., Hermann, K.M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., Hassabis, D.: Hybrid computing using a neural network with dynamic external memory. Nature 538(7626), 471–476 (2016)
Joulin, A., Mikolov, T.: Inferring algorithmic patterns with stack-augmented recurrent nets. In: NIPS, pp. 190–198 (2015)
Neelakantan, A., Le, Q.V., Sutskever, I.: Neural programmer: inducing latent programs with gradient descent. CoRR abs/1511.04834 (2015)
Alur, R., Bodík, R., Dallal, E., Fisman, D., Garg, P., Juniwal, G., Kress-Gazit, H., Madhusudan, P., Martin, M.M.K., Raghothaman, M., Saha, S., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. In: Dependable Software Systems Engineering, pp. 1–25 (2015)
Gulwani, S., Harris, W.R., Singh, R.: Spreadsheet data manipulation using examples. Commun. ACM 55(8), 97–105 (2012)
Alur, R., Radhakrishna, A., Udupa, A.: Scaling enumerative program synthesis via divide and conquer. In: TACAS, pp. 319–336 (2017)
Raza, M., Gulwani, S.: Automated data extraction using predictive program synthesis. In: AAAI, pp. 882–890 (2017)
Devlin, J., Uesato, J., Bhupatiraju, S., Singh, R., Mohamed, A., Kohli, P.: Robustfill: neural program learning under noisy I/O. In: ICML (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Singh, R., Gulwani, S.: Predicting a correct program in programming by example. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 398–414. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21690-4_23
Menon, A.K., Tamuz, O., Gulwani, S., Lampson, B.W., Kalai, A.: A machine learning framework for programming by example. In: Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16–21 June 2013, pp. 187–195 (2013)
Balog, M., Gaunt, A.L., Brockschmidt, M., Nowozin, S., Tarlow, D.: Deepcoder: learning to write programs. In: ICLR (2017)
Singh, R.: Blinkfill: semi-supervised programming by example for syntactic string transformations. PVLDB 9(10), 816–827 (2016)
Ellis, K., Gulwani, S.: Learning to learn programs from examples: going beyond program structure. In: IJCAI, pp. 1638–1645 (2017)
Mayer, M., Soares, G., Grechkin, M., Le, V., Marron, M., Polozov, O., Singh, R., Zorn, B.G., Gulwani, S.: User interaction models for disambiguation in programming by example. In: UIST, pp. 291–301 (2015)
Padhi, S., Jain, P., Perelman, D., Polozov, O., Gulwani, S., Millstein, T.: Flashprofile: interactive synthesis of syntactic profiles. arXiv preprint arXiv:1709.05725 (2017)
Jha, S., Gulwani, S., Seshia, S.A., Tiwari, A.: Oracle-guided component-based program synthesis. In: ICSE, pp. 215–224 (2010)
Gulwani, S., Marron, M.: Nlyze: interactive programming by natural language for spreadsheet data analysis and manipulation. In: SIGMOD, pp. 803–814 (2014)
Le, V., Gulwani, S., Su, Z.: Smartsynth: synthesizing smartphone automation scripts from natural language. In: The 11th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys), pp. 193–206 (2013)
Raza, M., Gulwani, S., Milic-Frayling, N.: Compositional program synthesis from natural language and examples. In: IJCAI, pp. 792–800 (2015)
Solar-Lezama, A.: Program Synthesis by Sketching. Ph.D. thesis, University of California, Berkeley (2008)
Simpkins, C.: Integrating reinforcement learning into a programming language. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2010, Atlanta, Georgia, USA, July 11–15, 2010 (2010)
Bielik, P., Raychev, V., Vechev, M.T.: Programming with “big code”: Lessons, techniques and applications. In: 1st Summit on Advances in Programming Languages, SNAPL 2015, May 3–6, 2015, Asilomar, California, USA, pp. 41–50 (2015)
Feser, J.K., Brockschmidt, M., Gaunt, A.L., Tarlow, D.: Neural functional programming. CoRR abs/1611.01988 (2016)
Singh, R., Kohli, P.: AP: artificial programming. In: 2nd Summit on Advances in Programming Languages, SNAPL 2017, May 7–10, 2017, Asilomar, CA, USA, pp. 16:1–16:12 (2017)
Acknowledgments
We thank Nagarajan Natarajan, Naren Datha, Danny Simmons, Abhishek Mohta, Alex Polozov, and Daniel Perelman for their participation in the ongoing unpublished work that we have described in this article related to using ML techniques for search and ranking. We thank the entire PROSE team (https://microsoft.github.io/prose/team/) that has been developing and productizing the PBE technology inside many Microsoft products while also inspiring relevant research problem definitions. We thank Sriram Rajamani, Rishabh Singh, and Joseph Sirosh for stimulating discussions related to the topics discussed in this article.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Gulwani, S., Jain, P. (2017). Programming by Examples: PL Meets ML. In: Chang, BY. (eds) Programming Languages and Systems. APLAS 2017. Lecture Notes in Computer Science(), vol 10695. Springer, Cham. https://doi.org/10.1007/978-3-319-71237-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-71237-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-71236-9
Online ISBN: 978-3-319-71237-6
eBook Packages: Computer ScienceComputer Science (R0)