Abstract
In the era of frozen pre-train model weights and fine-tuning large language models (LLMs) with prompts, we find that when using LLMs with standard prompt templates for text style transfer (TST), without limitations on the semantic space and sufficient context information, the model may generate text that deviates from the target style. We propose a set of new prompt templates integrated into a novel framework for arbitrary text style transfer, which balances transfer strength and fluency to enhance the accuracy of large language models in performing TST. Achieving an impressive 94.0% accuracy in transfer strength using GPT-4, our framework demonstrates significant performance. It also enables GPT-3.5-Turbo to surpass the performance of GPT-4 with the standard prompt. Additionally, due to the issue with unreliable TST metrics, we propose a novel prompt for TST evaluation. This prompt integrates scores from transfer strength, content retention, and fluency into a single score. We use this prompt to reevaluate previous TST models and highlight significant progress of our framework. Finally, we discover score fluctuation when using LLMs for text evaluation and propose an approach that requires LLMs to provide explanations. It enhances the evaluation stability by over 13% compared to prompts that do not have this requirement.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vineet, J., et al.: Disentangled representation learning for non-parallel text style transfer. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 424–434. Association for Computational Linguistics, Florence, Italy (2019)
Di, J., et al.: Deep learning for text style transfer: a survey. Comput. Linguist. 48(1), 155–205 (2022)
Martina, T., Sonja, G.: A review of text style transfer using deep learning. IEEE Trans. Artif. Intell. 3(5), 669–684 (2022)
Yupeng, C., et al.: A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol. (2024)
Phil, O., et al.: A call for standardization and validation of text style transfer evaluation. In: Findings of the Association for Computational Linguistics: ACL 2023, pp. 10791–10815. Association for Computational Linguistics, Toronto, Canada (2023)
Matt, P.: A call for clarity in reporting BLEU scores. In: Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 186–191. Association for Computational Linguistics, Brussels, Belgium (2018)
Mirac, S., Luke, M., Dan, J.: Prompt-and-rerank: a method for zero-shot and few-shot arbitrary textual style transfer with small language models. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 2195–2222. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates (2022)
Huiyuan, L., Antonio, T., Malvina, N.: Multidimensional evaluation for text style transfer using ChatGPT (2023). arXiv:2304.13462
Tianyi, Z., et al.: Bertscore: evaluating text generation with BERT (2019). arXiv:1904.09675
Xiang, Z., Junbo, Z., Yann, L.: Character-level convolutional networks for text classification. In: Proceedings of the 28th International Conference on Neural Information Processing Systems – Volume 1, NIPS 2015, pp. 649–657. MIT Press, MA, USA (2015)
Ruining, H., Julian, M.: Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th International Conference on World Wide Web, WWW’16, pp. 507–517. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE (2016)
Sudha, R., Joel, T.: Dear sir or madam, may I introduce the GYAFC dataset: corpus, benchmarks and metrics for formality style transfer. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 129–140. Association for Computational Linguistics, New Orleans, Louisiana (2018)
Yinhan, L., et al.: Roberta: a robustly optimized BERT pretraining approach (2019). arXiv:1907.11692
Victor, S., et al.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. Neural Inf. Process. Syst. (2019)
Alec, R., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
Shrimai, P., et al.: Style transfer through back-translation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 866–876. Association for Computational Linguistics, Melbourne, Australia (2018)
Tianxiao, S., et al.: Style transfer from non-parallel text by cross-alignment. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 6833–6844. Curran Associates Inc., NY, USA (2017)
Juncen, L., et al.: Delete, retrieve, generate: a simple approach to sentiment and style transfer. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1865–1874. Association for Computational Linguistics, New Orleans, Louisiana (2018)
Akhilesh, S., et al.: “Transforming” delete, retrieve, generate approach for controlled text style transfer. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3269–3279. Association for Computational Linguistics, Hong Kong, China (2019)
Fuli, L., et al.: A dual reinforcement learning framework for unsupervised text style transfer. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 5116–5122. International Joint Conferences on Artificial Intelligence Organization, Macao, China (2019)
Zhenxin, F., et al.: Style transfer in text: exploration and evaluation. In: Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), pp. 1–2. Association for the Advancement of Artificial Intelligence, New Orleans, Louisiana, USA (2018)
Yixin, L., Graham N., John W.: On learning text style transfer with direct rewards. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4262–4273. Association for Computational Linguistics, Online (2021)
Emily, R., et al.: A recipe for arbitrary text style transfer with large language models. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 837–848. Association for Computational Linguistics, Dublin, Ireland (2022)
Guoqing, L., et al.: Prompt-based editing for text style transfer. In: Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 5740–5750. Association for Computational Linguistics, Singapore (2023)
ZHIPU:Zhipu ai devday glm-4 (2024). https://zhipuai.cn/devday
Angela, F., Mike L., Yann D.: Hierarchical neural story generation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 889–898. Association for Computational Linguistics, Melbourne, Australia (2018)
This study was funded by National Natural Science Foundation of China Research Project (grant number 62076103), Guangdong Basic and Applied Basic Research Project (grant number 2021A1515011171) and Guangzhou Basic Research Plan, Basic and Applied Basic Research Project (grant number 202102080282).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zeng, B. et al. (2024). THE BAT: Thoughts Hierarchical Enhancement Beyond Arbitrary Text Style Transfer. In: Huang, DS., Si, Z., Zhang, C. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science(), vol 14878. Springer, Singapore. https://doi.org/10.1007/978-981-97-5672-8_32
Download citation
DOI: https://doi.org/10.1007/978-981-97-5672-8_32
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5671-1
Online ISBN: 978-981-97-5672-8
eBook Packages: Computer ScienceComputer Science (R0)