Skip to main content
SpringerLink
Log in

Optimization of Spirulina sp. cultivation using reinforcement learning with state prediction based on LSTM neural network

  • Published:
Journal of Applied Phycology Aims and scope Submit manuscript

Abstract

Spirulina/Arthrospira is well known as a microscopic filamentous cyanobacterium, which is rich in nutrition and minerals. Therefore, Spirulina/Arthrospira cultivation industry has been adopting many optimization methods for enhancing biomass productivity. Recently, machine learning has been an emerging approach thanks to their efficacy. In this paper, a novel time-dependent reinforcement learning (RL) method with state prediction by long short-term memory (LSTM) networks is proposed to optimize the dry-weight yield of Spirulina sp. HH cultivation. The simulation results show that under the same light condition, with the proposed algorithm, the Spirulina sp. HH cultivation system produces a yield 17% higher than that of the traditional cultivation method and 10% higher than that of the threshold-based method. The results of the RL method from this study promise a significant benefit in Spirulina farm production when applying it for proactively planning biomass production and enhancing profit.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of data and material

Ready to present if requested

Code availability

We used the Python programming language. Code will be provided upon request

Notes

  1. The commercially produced species have been transferred from the genus Spirulina to the genus Arthrospira; however we will continue to use the name Spirulina in the text.

References

  • Aiba S (1982) Growth kinetics of photosynthetic microorganisms. In: Microbial reactions. Advances in Biochemical Engineering, 23rd edn. Springer, Berlin, pp 85–156

    Chapter  Google Scholar 

  • Béchet Q, Shilton A, Guieysse B (2013) Modeling the effects of light and temperature on algae growth: state of the art and critical assessment for productivity prediction during outdoor cultivation. Biotechnol Adv 31:1648–1663

    Article  Google Scholar 

  • Bellman R (1957) A Markovian decision process. J Math Mech 6:679–684

    Google Scholar 

  • Boyan J, Littman M (2000) Exact solutions to time-dependent MDPs. Adv Neural Inf Proces Syst 13:1026–1032

    Google Scholar 

  • Choi E, Hewlett D, Uszkoreit J, Polosukhin I, Lacoste A, Berant J (2017) Coarse-to-fine question answering for long documents. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp 209–220

  • Chung I, Kim S, Lee J, Kim KJ, Hwang SJ, Yang E (2020) Deep mixed effect model using Gaussian processes: a personalized and reliable prediction for healthcare. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. pp 3649–3657

  • da Fontoura PD, Duarte JH, Vendruscolo RG, Wagner R, Ballus CA, da Silva OW, Godoy HT, Barcia MT, de Morais MG, Radmann EM, Costa JAV (2020) Role of light emitting diode (LED) wavelengths on increase of protein productivity and free amino acid profile of Spirulina sp. cultures. Bioresour Technol 306:123184

    Article  Google Scholar 

  • De Morais MG, Da Fontoura PD, Moreira JB, Duarte JH, Costa JAV (2018) Phycocyanin from microalgae: properties, extraction and purification, with some recent applications. Ind Biotechnol 14:30–37

    Article  Google Scholar 

  • Doan YTT, Obbard JP (2015) Two-stage cultivation of a Nannochloropsis mutant for biodiesel feedstock. J Appl Phycol 27:2203–2208

    Article  CAS  Google Scholar 

  • Fan H, Wang K, Wang C, Yu F, He X, Ma J, Li X (2020) A comparative study on growth characters and nutrients removal from wastewater by two microalgae under optimized light regimes. Environ Technol Innov 19:100849

    Article  Google Scholar 

  • Gershwin ME, Belay A (eds) (2007) Spirulina in human nutrition and health. CRC Press, Boca Raton

    Google Scholar 

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780

    Article  CAS  Google Scholar 

  • Huong TT, Thanh NH, Van NT, Dat NT, Long NV, Mrshall A (2018) Water and energy-efficient irrigation based on Markov decision model for precision agriculture. In: 2018 IEEE Seventh International Conference on Communications and Electronics (ICCE). IEEE, pp 51–56

  • Irukula S (2015) Reinforcement learning based controller for precision irrigation. Masters Thesis, Texas A&M University

  • Jeong K-S, Joo G-J, Kim H-W, Ha K, Rechnagel F (2001) Prediction and elucidation of phytoplankton dynamics in the Nakdong River (Korea) by means of a recurrent artificial neural network. Ecol Model 146:115–129

    Article  CAS  Google Scholar 

  • Jin J, Song C, Li H, Gai K, Wang J, Zhang W (2018) Real-time bidding with multi-agent reinforcement learning in display advertising. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management. pp 2193–2201

  • Juneja A, Murthy GS (2018) Model predictive control coupled with economic and environmental constraints for optimum algal production. Bioresour Technol 250:556–563

    Article  CAS  Google Scholar 

  • Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations,. ICLR 2015 - Conference Track Proceedings. arXiv:1412.6980

  • Kormushev P, Calinon S, Caldwell DG (2013) Reinforcement learning in robotics: applications and real-world challenges. Robotics 2:122–148

    Article  Google Scholar 

  • Landeta JMI, Cortés CBY, Azúa HM (2014) Markovian decision process to find optimal policies in the management of an orange farm. Investig Operacional 35:68–77

    Google Scholar 

  • Lee HY, Erickson LE, Yang SS (1987) Kinetics and bioenergetics of light-limited photoautotrophic growth of Spirulina platensis. Biotechnol Bioeng 29:832–843

    Article  CAS  Google Scholar 

  • Lee E, Jalalizadeh M, Zhang Q (2015) Growth kinetic models for microalgae cultivation: a review. Algal Res 12:497–512

    Article  Google Scholar 

  • Liang X, Wang T, Yang L, Xing E (2018) CIRL: controllable imitative reinforcement learning for vision-based self-driving. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol 11211. Springer, Cham pp 604-620

  • Liu L, Sukhatme GS (2018) A solution to time-varying Markov decision processes. IEEE Robot Autom Lett 3:1631–1638

    Article  Google Scholar 

  • Ma Y, Kelman A, Daly A, Borrelli F (2012) Predictive control for energy efficient buildings with thermal storage: modeling, stimulation, and experiments. IEEE Control Syst Mag 32:44–64

    CAS  Google Scholar 

  • Müller MA, Allgöwer F (2017) Economic and distributed model predictive control: recent developments in optimization-based control. SICE J Control Meas Syst Integr 10:39–52

    Article  Google Scholar 

  • Muys M, Sui Y, Schwaiger B, Lesueur B, Vandesheuvel D, Vermeir P, Vlawminck SE (2019) High variability in nutritional value and safety of commercially available Chlorella and Spirulina biomass indicates the need for smart production strategies. Bioresour Technol 275:247–257

  • Nwoba EG, Parlevliet DA, Laird DW, Alameh K, Moheimani NR (2019) Light management technologies for increasing algal photobioreactor efficiency. Algal Res 39:101433

    Article  Google Scholar 

  • Ogbonna JC, Tanaka H (2000) Light requirement and photosynthetic cell cultivation -development of processes for efficient light utilization in photobioreactors. J Appl Phycol 12:207–218

    Article  Google Scholar 

  • Pan X, You Y, Wang Z, Lu C (2017) Virtual to real reinforcement learning for autonomous driving. arXiv 170403952

  • Paulus R, Xiong C, Socher R (2017) A deep reinforced model for abstractive summarization. arXiv170504304

  • Rasmussen C.E. (2004) Gaussian processes in machine learning. In: Bousquet O., von Luxburg U., Rätsch G. (eds) Advanced Lectures on Machine Learning. ML 2003. Lecture Notes in Computer Science, vol 3176. Springer, Berlin pp 63–71

  • Rastogi RP, Madamwar D, Pandey A (2017) Nutraceuticals from algae and cyanobacteria. In: Rastogi RP, Madamwar D, Pandey A (eds) Algal Green Chemistry: Recent Progress in Biotechnology. Elsevier, Amsterdam, pp 66–89

    Google Scholar 

  • Ravindran B, Gupta SK, Cho W-M, Kim JK, Lee SR, Jeong K-H, Lee DJ, Choi H-C (2016) Microalgae potential and multiple roles-current progress and future prospects-an overview. Sustainability 8:1215

    Article  Google Scholar 

  • Rostam M, Nagamune R, Grebenyuk V (2020) A hybrid Gaussian process approach to robust economic model predictive control. J Process Control 92:149–160

    Article  CAS  Google Scholar 

  • Salama E-S, Hwang J-H, El-Dalatony MM, Kurade MB, Kabra AN, Abou-Shanab RAI, Kim K-H, Yang I-S, Govindwar SP, Kim S, Jeon B-H (2018) Enhancement of microalgal growth and biocomponent-based transformations for improved biofuel recovery: a review. Bioresour Technol 258:365–375

    Article  CAS  Google Scholar 

  • Schaefer AJ, Bailey MD, Shechter SM, Roberts MS (2005) Modeling medical treatment using Markov decision processes. In: Brandeau ML, Sainfort F, Pierskalla WP (eds). Operations Research and Health Care. I Springer, Boston, pp 593–612

  • Schütze N, Schmitz GH (2007) Neuro-dynamic programming as a new framework for decision support for deficit irrigation systems. In: International Congress on Modelling and Simulation (ModSim 2007). Christchurch, New Zealand, pp 2271–2277

    Google Scholar 

  • Song H, Liu C-C, Lawarrée J, Dahlgren RW (2000) Optimal electricity supply bidding by Markov decision process. IEEE Trans Power Syst 15:618–624

    Article  Google Scholar 

  • Soni RA, Sudhakar K, Rana RS (2017) Spirulina – from growth to nutritional product: a review. Trends Food Sci Technol 69:157–171

    Article  CAS  Google Scholar 

  • Sun L, Yang Y, Hu J, Porter D, Marek T, Hillyer C (2017) Reinforcement learning control for water-efficient agricultural irrigation. In: 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC). IEEE, pp 1334–1341

  • Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  • Syafiie S, Tadeo F, Martinez E (2007) Model-free learning control of neutralization processes using reinforcement learning. Eng Appl Artif Intell 20:767–782

    Article  Google Scholar 

  • Talib A, Recknagel F, Cao H, van der Molen DT (2008) Forecasting and explanation of algal dynamics in two shallow lakes by recurrent artificial neural network and hybrid evolutionary algorithm. Math Comput Simul 78:424–434

    Article  Google Scholar 

  • Vonshak A (2002) Spirulina: growth, physiology and biochemistry. In: Avigad Vonshak (ed) Spirulina platensis (Arthrospira): Physiology, cell-biology and biotechnology. Taylor&Francis, London pp 43–66

  • Vu MTT, Douëtte C, Rayner TA, Thoisen C, Nielsen SL, Hansen BW (2016) Optimization of photosynthesis, growth, and biochemical composition of the microalga Rhodomonas salina—an established diet for live feed copepods in aquaculture. J Appl Phycol 28:1485–1500

    Article  CAS  Google Scholar 

  • Walter M, Recknagel F, Carpenter C, Bormans M (2001) Predicting eutrophication effects in the Burrinjuck Reservoir (Australia) by means of the deterministic model SALMO and the recurrent neural network model ANNA. Ecol Model 146:97–113

    Article  CAS  Google Scholar 

  • Wang Y, Alamo T, Puig V, Cembrano G (2018) Economic model predictive control with nonlinear constraint relaxation for the operational management of water distribution networks. Energies 11:991

    Article  Google Scholar 

  • Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8:279–292

    Google Scholar 

  • Wholesale Electricity Tariff (2018) Access 2018 URL: https://en.evn.com.vn/d6/news/WHOLESALE-ELECTRICITYTARIFF-9-28-260.aspx

  • Wood AM, Everroad RC, Wingard LM (2005) Measuring growth rates in microalgal cultures. In: Andersen RA (ed) Algal Culturing Techniques. Elsevier, Amsterdam, pp 269–288

    Google Scholar 

  • Zhi PW, Zhao Y (2005) Morphological reversion of Spirulina (Arthrospira) platensis (Cyanophyta): from linear to helical. J Phycol 41:622–628

    Article  Google Scholar 

Download references

Funding

This research is funded by the Hanoi University of Science and Technology (HUST) under grant number T2017-LN-01.

Author information

Authors and Affiliations

  1. School of Environmental Science and Technology, Hanoi University of Science and Technology, 1 Dai Co Viet, Hanoi, Vietnam

    Yen Thi-Thai Doan

  2. School of Electronics and Telecommunications, Hanoi University of Science and Technology, 1 Dai Co Viet, Hanoi, Vietnam

    Minh-Tri Ho, Huu-Kim Nguyen & Huy-Dung Han

Authors
  1. Yen Thi-Thai Doan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minh-Tri Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huu-Kim Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huy-Dung Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Doan Thi Thai Yen made substantial contributions to:

• Obtaining of funding

• Conception and design

• Collection and assembly of the data

• Analysis and interpretation of the data

• Drafting of the article

• Critical revision of the article for important intellectual content

• Final approval of the final manuscript

Ho Minh Tri

• Conception and design

• Simulation and performance analysis of the algorithm

• Collection and assembly of the data

• Drafting of the first manuscript

• Final approval of the final manuscript

Nguyen Huu Kim

• Development of the algorithm

• Collection and assembly of the data

• Drafting of the article

• Final approval of the final manuscript

Han Huy Dung

• Conception and design

• Editing and revision of the manuscript

• Final approval of the final manuscript

Doan Thi Thai Yen took overall responsibility for the integrity of the work as a whole, from inception to the finished article.

Corresponding authors

Correspondence to Yen Thi-Thai Doan or Huy-Dung Han.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doan, Y.TT., Ho, MT., Nguyen, HK. et al. Optimization of Spirulina sp. cultivation using reinforcement learning with state prediction based on LSTM neural network. J Appl Phycol 33, 2733–2744 (2021). https://doi.org/10.1007/s10811-021-02488-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10811-021-02488-y

Keywords

Search

Navigation