Abstract
This research investigates the role of prompt engineering in enhancing the performance and generalisation of large-scale language models (LLMs) across a wide range of Natural Language Processing (NLP) tasks. The study introduces a comprehensive framework for prompt engineering, titled the “PERFECT” framework, and evaluates its effectiveness across different tasks and domains. The research findings underscore the pivotal role of advanced prompting techniques in eliciting more nuanced and flexible responses from AI models. The study also explores the future implications of prompt engineering, including the integration of reinforcement learning with human feedback, the emergence of prompt engineering as a new job market, and the rise of context-aware and interactive prompts. The research contributes to a deeper understanding of the principles, mechanisms, and best practices in prompt engineering, with practical implications for improving LLM performance and reducing the barrier to entry for new adoptees through using prompting frameworks. The research aims have been largely achieved, providing a new framework for prompting while also exploring future advancements. However, the study also highlights the need for further exploration of the constraints placed on current prompting techniques, such as token size and context window.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, S.F., Beeferman, D., Rosenfeld, R.: Evaluation metrics for language models (2008). https://doi.org/10.1184/R1/6605324.v1
Chen, L., Zaharia, M., Zou, J.: How is ChatGPT’s behavior changing over time? (2023). https://doi.org/10.48550/arXiv.2307.09009
Cobbe, K., et al.: Training verifiers to solve math word problems (2021). https://doi.org/10.48550/arXiv.2110.14168
Cummins, R., Paik, J.H., Lv, Y.: A Pólya urn document language model for improved information retrieval. ACM Trans. Inf. Syst. 33(4), 1–34 (2015). https://doi.org/10.1145/2746231
Brown, T.B., Mann, B., Ryder, N.: Language models are few-shot learners (2020). arXiv preprint arXiv:2005.14165
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 4171–4186 (2019). https://aclanthology.org/N19-1423/
Geva, M., Zhao, Y., Lu, Y., Li, Z., Dong, L., Sun, H.: Tree of thoughts: deliberate problem solving with language models. arXiv preprint arXiv:2305.10601 (2023).https://arxiv.org/pdf/2305.10601.pdf
Kojima, T., Gu, S S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large Language models are zero-shot reasoners (2022). https://arxiv.org/abs/2205.11916
Liu, J., Cui, L., Liu, H., Huang, D., Wang, Y., Zhang, Y.: Logiqa: a challenge dataset for machine reading comprehension with logical reasoning. arXiv preprint (2020). https://arxiv.org/abs/2007.08124
Lu, J., Li, C., Niu, S., Zhou, M.: Multimodal chain-of-thought reasoning for visual question answering. arXiv preprint arXiv:2302.00923 (2023). https://arxiv.org/pdf/2302.00923.pdf
Petroni, F., Piktus, A., Gupta, N., Schlichtkrull, M., Lewis, M., Riedel, S.: How context affects language models’ factual predictions. arXiv preprint arXiv:2102.08667 (2021). https://arxiv.org/abs/2102.08667
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI Blog (2018). https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020). https://jmlr.org/papers/v21/20-074.html
Rahi, S.: Research design and methods: a systematic review of research paradigms, sampling issues and instruments development. Int. J. Econ. Manag. Sci. 6, 403 (2017). https://www.researchgate.net/publication/316701205_Research_Design_and_Methods_A_Systematic_Review_of_Research_Paradigms_Sampling_Issues_and_Instruments_Development
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models (2023). https://doi.org/10.48550/arXiv.2201.11903
Wei, Z., Zhao, Y., Lu, Y., Li, Z., Dong, L., Sun, H.: Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv preprint arXiv:2203.11171 (2023). https://arxiv.org/pdf/2203.11171.pdf
Yang, H., Yue, S., He, Y.: Auto-GPT for online decision making: benchmarks and additional opinions. arXiv:2306.02224 (2023)
Zhang, J., Liu, Z., Xiong, C., Sun, M., Zhou, M., Gao, J.: KOSMOS: a universal system for multimodal perception, language understanding, and instruction following. arXiv preprint arXiv:2302.14045 (2023). https://arxiv.org/pdf/2302.14045.pdf
Zhao, J., Lu, K., Chen, H.: Learning to Prompt for Vision-Language Models. arXiv preprint arXiv:2108.13348 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ratnayake, H., Wang, C. (2024). A Prompting Framework to Enhance Language Model Output. In: Liu, T., Webb, G., Yue, L., Wang, D. (eds) AI 2023: Advances in Artificial Intelligence. AI 2023. Lecture Notes in Computer Science(), vol 14472. Springer, Singapore. https://doi.org/10.1007/978-981-99-8391-9_6
Download citation
DOI: https://doi.org/10.1007/978-981-99-8391-9_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8390-2
Online ISBN: 978-981-99-8391-9
eBook Packages: Computer ScienceComputer Science (R0)