Abstract
One of the ways Large Language Models (LLMs) are used to perform machine learning tasks is to provide them with a few examples before asking them to produce a prediction. This is a meta-learning process known as few-shot learning. In this paper, we use available Search-Based methods to optimise the number and combination of examples that can improve an LLM’s estimation performance, when it is used to estimate story points for new agile tasks. Our preliminary results show that our SBSE technique improves the estimation performance of the LLM by 59.34% on average (in terms of mean absolute error of the estimation) over three datasets against a zero-shot setting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
User stories, software tasks, and issues are used interchangeably in this paper.
References
Chen, L., Zaharia, M., Zou, J.: FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176 (2023)
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)
Grigelionis, B.: Student’s t-distribution and related stochastic processes. Springer (2013)
Hill, G.W.: Algorithm 396: Student’s t-quantiles. Commun. ACM 13(10), 619–620 (1970)
OpenAI: GPT-4 technical report (2023)
Ouyang, S., Zhang, J.M., Harman, M., Wang, M.: LLM is like a box of chocolates: the non-determinism of ChatGPT in code generation. arXiv preprint arXiv:2308.02828 (2023)
Tawosi, V., Al-Subaihin, A., Moussa, R., Sarro, F.: A versatile dataset of agile open source software projects. In: Proceedings of the 19th International Conference on Mining Software Repositories, pp. 707–711 (2022)
Tawosi, V., Al-Subaihin, A., Sarro, F.: Investigating the effectiveness of clustering for story point estimation. In: 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 827–838. IEEE (2022)
Tawosi, V., Moussa, R., Sarro, F.: Agile effort estimation: have we solved the problem yet? insights from a replication study. IEEE Trans. Software Eng. 49(4), 2677–2697 (2022)
Tawosi, V., Sarro, F., Petrozziello, A., Harman, M.: Multi-objective software effort estimation: a replication study. IEEE Trans. Softw. Eng. 48(8), 3185–3205 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tawosi, V., Alamir, S., Liu, X. (2024). Search-Based Optimisation of LLM Learning Shots for Story Point Estimation. In: Arcaini, P., Yue, T., Fredericks, E.M. (eds) Search-Based Software Engineering. SSBSE 2023. Lecture Notes in Computer Science, vol 14415. Springer, Cham. https://doi.org/10.1007/978-3-031-48796-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-48796-5_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48795-8
Online ISBN: 978-3-031-48796-5
eBook Packages: Computer ScienceComputer Science (R0)