Abstract
This chapter explores the emerging discipline of LLMOps, contrasting it with traditional MLOps approaches and highlighting unique considerations when operationalizing GenAI models and applications. A detailed examination of implementing LLMOps across the model lifecycle is provided, encompassing activities like base model selection, prompt engineering, model tuning, deployment, and monitoring. Recognizing security as a critical priority, strategies for integrating DevSecOps into LLMOps are outlined, establishing security as a shared responsibility across the development and operational lifecycle. The chapter offers conceptual foundations and practical guidance for successfully navigating the intricacies of LLMOps.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brownlee, J. (2017, December 20). A gentle introduction to transfer learning for deep learning MachineLearningMastery.com. Machine Learning Mastery. Retrieved August 30, 2023, from https://machinelearningmastery.com/transfer_learning_for_deep_learning/
Collins, M. (2023). Automate ML model retraining and deployment with MLflow in Databricks. Towards Data Science. Retrieved August 30, 2023, from https://towardsdatascience.com/automate_ml_model_retraining_and_deployment_with_mlflow_in_databricks ad29f6146f80
Gamble, M., Chen, M., Cassel, D., Grygleski, M., Lawson, L., MacManus, R., Branscombe, M., Taft, D. K., Udell, J., Myers, J., Ferguson, S., Hall, S., Cameron, B., Joslyn, H., Kimani, R., Benny, S., Gupta, P., Tigli, U., Flora, D., Melamed, D. (2023, July 6). LlamaIndex and the New World of LLM Orchestration Frameworks. The New Stack. Retrieved August 30, 2023, from https://thenewstack.io/llamaindex_and_the_new_world_of_llm_orchestration_frameworks/
Ganesan, K. (2017, January 26). An intro to ROUGE, and how to use it to evaluate summaries. freeCodeCamp. Retrieved August 30, 2023, from https://www.freecodecamp.org/news/what_is_rouge_and_how_it_works_for_evaluation_of_summaries_e059fb8ac840/
Greenberg, K. (2023, March 24). DevSecOps puts security in the software cycle. TechRepublic. Retrieved August 30, 2023, from https://www.techrepublic.com/article/devsecops_security_software_cycle/
Huyen, C. (2023, May 2). RLHF: Reinforcement learning from human feedback. Chip Huyen. Retrieved August 30, 2023, from https://huyenchip.com/2023/05/02/rlhf.html
Kerner, S. M. (2023, April 21). Weights and biases debuts LLMOps tools to support prompt engineers. VentureBeat. Retrieved August 30, 2023, from https://venturebeat.com/ai/weights_and_biases_debuts_llmops_tools_to_support_prompt_engineers/
Khandelwal, R. (2020, January 25). BLEU — Bilingual evaluation understudy | by RenuKhandelwal. Towards Data Science. Retrieved August 30, 2023, from https://towardsdatascience.com/bleu_bilingual_evaluation_understudy_2b4eab9bcfd1
Korstanje, J. (2021, August 31). The F1 score. Towards Data Science. Retrieved August 30, 2023, from https://towardsdatascience.com/the_f1_score_bec2bbc38aa6
Laaksonen, E. (2023). LLMOps: MLOps for large language models. Valohai. Retrieved August 30, 2023, from https://valohai.com/blog/llmops/
MSV, J. (2023, August 28). A brief guide to LangChain for software developers. InfoWorld. Retrieved August 30, 2023, from https://www.infoworld.com/article/3705097/a_brief_guide_to_langchain_for_software_developers.html
Ren, R. (2023, June 16). Surging demand for large language models fuels the meteoric rise of startup Dify. PingWest. Retrieved August 30, 2023, from https://en.pingwest.com/a/11852
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Huang, K., Manral, V., Wang, W. (2024). From LLMOps to DevSecOps for GenAI. In: Huang, K., Wang, Y., Goertzel, B., Li, Y., Wright, S., Ponnapalli, J. (eds) Generative AI Security. Future of Business and Finance. Springer, Cham. https://doi.org/10.1007/978-3-031-54252-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-54252-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54251-0
Online ISBN: 978-3-031-54252-7
eBook Packages: Business and ManagementBusiness and Management (R0)