Skip to main content
SpringerLink
Log in

Advanced ensemble machine-learning and explainable ai with hybridized clustering for solar irradiation prediction in Bangladesh

  • Research
  • Published:
Theoretical and Applied Climatology Aims and scope Submit manuscript

Abstract

The solar revolution in Bangladesh stands as a symbol of hope and self-reliance, illuminating communities and steering the nation towards a more sustainable future. Highlighting the crucial importance of solar irradiance forecasting in attaining sustainable energy objectives, this study utilizes advanced machine learning techniques to enlighten solar irradiance prediction in Bangladesh. Ensemble machine learning has become a vital tool that significantly enhances the precision and reliability of solar radiation predictions by amalgamating diverse model outputs, providing a more robust and accurate forecasting frame- work. Previous research efforts have overlooked the exploration of this diverse range in solar radiation prediction using ensemble machine learning. This study addresses this gap by conducting a detailed experiment on ensemble artificial intelligence techniques. Furthermore, the research introduces Explain- able AI (XAI) with ensemble machine learning, shedding light on factors influencing predictions and providing valuable insights for decision-makers in the solar energy sector. Notably, the study pioneers an XAI-based analysis specific to Bangladesh, marking a significant stride in solar radiation predic- tion. Additionally, a novel hybridized approach incorporating various clustering techniques and the LightGBM algorithm is introduced, offering an efficient framework for solar radiation prediction. As a result, this study contributes to our understanding and optimization of solar irradiance prediction by offer- ing a comprehensive method that integrates XAI, ensemble approaches, and machine learning. We have developed an autoML tool based on XAI and Ensem- ble as a further contribution. We have validated our result with the low-code PyCaret machine learning package to see that, among all the methods, lightGBM has shown promising results in terms of solar irradiance prediction. Ensem- ble machine learning boosting techniques, notably LightGBM and CatBoost, exhibited superior performance, achieving high R2 scores of 0.91 and demonstrat- ing remarkable accuracy. Through feature importance analysis, it was discerned that”Sunshine (Hours)” emerged as the most impactful factor influencing the predictive models. Employing a hybridized clustering approach, incorporating K- means-LightGBM, MiniBatchKMeans-LightGBM, Fuzzy C-Means-LightGBM, and GaussianMixture-LightGBM models, excelled in forecasting solar radiation into distinct clusters, particularly excelling in the”Very Cloudy” category.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Algorithm 2
Algorithm 3
Fig. 6
Algorithm 4
Fig. 7
Algorithm 5
Fig. 8
Algorithm 6
Fig. 9
Algorithm 7
Fig. 10
Algorithm 8
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

None.

References

  • Adams J, Hagras H (2020) A type-2 fuzzy logic approach to explainable ai for reg- ulatory compliance, fair customer outcomes and market stability in the global financial sector. In: 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp 1–8. IEEE

  • Alam MS, Al-Ismail FS, Hossain MS, Rahman SM (2023) Ensemble machine- learning models for accurate prediction of solar irradiation in bangladesh. Processes 11(3):908

    Article  Google Scholar 

  • Ali M (2020) Pycaret: An open source, low-code machine learning library in python. PyCaret version 2

  • Amin MN, Iftikhar B, Khan K, Javed MF, AbuArab AM, Rehman MF (2023) Prediction model for rice husk ash concrete using ai approach: Boosting and bagging algorithms. In: Structures, vol 50, pp 745–757. Elsevier

  • Arrieta AB, D´ıaz-Rodr´ıguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garc´ıa S, Gil-L´opez S, Molina D, Benjamins R et al (2020) Explainable artifi- cial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Inf Fusion 58:82–115

  • Ayodele TR, Ogunjuyigbe ASO, Amedu A, Munda JL (2019) Prediction of global solar irradiation using hybridized k-means and support vector regression algorithms. Renew Energy Focus 29:78–93. https://doi.org/10.1016/j.ref.2019.03.003

    Article  Google Scholar 

  • Bae KY, Jang HS, Sung DK (2016) Hourly solar irradiance prediction based on support vector machine and its error analysis. IEEE Trans Power Syst 32(2):935–945

    Google Scholar 

  • Bahani K, Ali-Ou-Salah H, Moujabbir M, Oukarfi B, Ramdani M (2020) A novel interpretable model for solar radiation prediction based on adaptive fuzzy clus- tering and linguistic hedges. In: Proceedings of the 13th international conference on intelligent systems: theories and applications, pp 1–6

  • Bezdek JC, Ehrlich R, Full W (1984) Fcm: The fuzzy c-means clustering algorithm. Comput Geosci 10(2–3):191–203

    Article  Google Scholar 

  • Biau G, Scornet E (2016) A random forest guided tour. TEST 25:197–227

    Article  Google Scholar 

  • Biparva D, Materassi D (2023) Interpretation of explainable ai methods as identification of local linearized models. IFAC-PapersOnLine 56(2):2383–2388. https://doi.org/10.1016/j.ifacol.2023.10.1211. 22nd IFAC World Congress

  • Cannon RL, Dave JV, Bezdek JC (1986) Efficient implementation of the fuzzy c- means clustering algorithms. IEEE Trans Pattern Anal Mach Intell 2:248–255

    Article  Google Scholar 

  • Chadaga K, Prabhu S, Bhat V, Sampathila N, Umakanth S, Chadaga R (2023) A decision support system for diagnosis of covid-19 from non-covid-19 influenza-like illness using explainable artificial intelligence. Bioengineering 10(4):439. https://doi.org/10.3390/bioengineering10040439

    Article  CAS  Google Scholar 

  • Chaibi M, Benghoulam EM, Tarik L, Berrada M, Hmaidi AE (2021) An inter- pretable machine learning model for daily global solar radiation prediction. Energies 14(21):7367

    Article  Google Scholar 

  • Chavan M, Patil A, Dalvi L, Patil A (2015) Mini batch k-means clustering on large dataset. Int J Sci Eng Technol Res 4(07):1356–1358

    Google Scholar 

  • Chen T, Guestrin C (2016) Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, pp 785–794

  • Dietterich TG et al (2002) Ensemble learning. The handbook of brain theory and neural networks 2(1):110–125

  • El-Sappagh S, Alonso JM, Islam SR, Sultan AM, Kwak KS (2021) A multi- layer multimodal detection and prediction model based on explainable artificial intelligence for alzheimer’s disease. Sci Rep 11(1):2660. https://doi.org/10.1038/s41598-021-82098-3

    Article  CAS  Google Scholar 

  • Frimane Â, Soubdhan T, Bright JM, Aggour M (2019) Nonparametric bayesian-based recognition of solar irradiance conditions: Applica- tion to the generation of high temporal resolution synthetic solar irradiance data. Solar Energy 182:462–479. https://doi.org/10.1016/j.solener.2019.02.052

    Article  Google Scholar 

  • Gillies S (2013) The shapely user manual. URL https://pypi.org/project/Shapely. Accessed 20 Aug 2023

  • Groß J (2003) Linear Regression vol 175. Springer

  • Hirata Y, Aihara K (2017) Improving time series prediction of solar irradiance after sunrise: Comparison among three methods for time series prediction. Solar Energy 149:294–301. https://doi.org/10.1016/j.solener.2017.04.020

    Article  Google Scholar 

  • Hissou H, Benkirane S, Guezzaz A, Azrour M, Beni-Hssane A (2023) A novel machine learning approach for solar radiation estimation. Sustainability 15(13):10609–10609. https://doi.org/10.3390/su151310609

    Article  Google Scholar 

  • http://apps.barc.gov.bd/climate/dashboard (2023) [Online; Accessed 2023–08–11]

  • Jay CB, Cockett JRB (1994) Shapely types and shape polymorphism. In: European Symposium on Programming, pp. 302–316. Springer

  • Kadiyala A, Kumar A (2018) Applications of python to evaluate the performance of bagging methods. Environ Prog Sustain Energy 37(5):1555–1559

    Article  CAS  Google Scholar 

  • Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu T-Y (2017) Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems 30

  • Khorasani M, Abdou M, Hern´andez Fern´andez J (2022) Streamlit basics. 31–62

  • Kushwah JS, Kumar A, Patel S, Soni R, Gawande A, Gupta S (2022) Comparative study of regressor and classifier with decision tree using mod- ern tools. Mater Today: Proc 56:3571–3576. https://doi.org/10.1016/j.matpr.2021.11.635. First International Conference on Design and Materials

  • Kuzlu M, Cali U, Sharma V, Guler O (2020) Gaining insight into solar photo- voltaic power generation forecasting utilizing explainable artificial intelligence tools. IEEE Access 8:187814–187823. https://doi.org/10.1109/ACCESS.2020.3031477

    Article  Google Scholar 

  • Kwon H, Park J, Lee Y (2019) Stacking ensemble technique for classifying breast cancer. Healthcare Inform Res 25(4):283–288

    Article  Google Scholar 

  • Lee Y, Oh J, Kim G (2020) Interpretation of load forecasting using explainable artifi- cial intelligence techniques. Trans Korean Inst Electr Eng 69(3):480–485

    Article  Google Scholar 

  • Liang X, Jacobucci R (2020) Regularized structural equation modeling to detect measurement bias: Evaluation of lasso, adaptive lasso, and elastic net. Struct Equ Model: Multidiscip J 27(5):722–734. https://doi.org/10.1080/10705511.2019.1693273

    Article  Google Scholar 

  • Marino DL, Wickramasinghe CS, Manic M (2018) An adversarial approach for explainable ai in intrusion detection systems. In: IECON 2018–44th Annual conference of the IEEE industrial electronics society, pp 3237–3243. IEEE

  • McCandless TC, Haupt SE, Young GS (2015) A model tree approach to forecasting solar irradiance variability. Solar Energy 120:514–524. https://doi.org/10.1016/j.solener.2015.07.020

    Article  Google Scholar 

  • McDonald GC (2009) Ridge regression. Wiley Interdiscip Rev: Comput Stat 1(1):93–100

    Article  Google Scholar 

  • Mishra DP, Jena S, Senapati R, Panigrahi A, Salkuti SR (2023) Global solar radiation forecast using an ensemble learning approach. Int J Power Electron Drive Syst 14(1):496–496. https://doi.org/10.11591/ijpeds.v14.i1.pp496-505

    Article  Google Scholar 

  • Myles AJ, Feudale RN, Liu Y, Woody NA, Brown SD (2004) An introduction to decision tree modeling. J Chemom 18(6):275–285

    Article  CAS  Google Scholar 

  • Natekin A, Knoll A (2013) Gradient boosting machines, a tutorial. Front Neurorobot 7:21

    Article  Google Scholar 

  • Pannu HS, Malhi A et al (2020) Deep learning-based explainable target classifica- tion for synthetic aperture radar images. In: 2020 13th International Conference on Human System Interaction (HSI), pp. 34–39. IEEE

  • Pavlyshenko B (2018) Using stacking approaches for machine learning models. In: 2018 IEEE second international conference on data stream mining & processing (DSMP), pp 255–258. IEEE

  • Peng K, Leung VC, Huang Q (2018) Clustering approach based on mini batch kmeans for intrusion detection system over big data. IEEE Access 6:11897–11906

    Article  Google Scholar 

  • Pierrot A, Goude Y (2011) Short-term electricity load forecasting with generalized additive models. Proceedings of ISAP power 2011

  • Polikar R (2012) Ensemble learning. Ensemble machine learning: Methods and applications 1–34

  • Prentzas N, Nicolaides A, Kyriacou E, Kakas A, Pattichis C (2019) Integrating machine learning with symbolic reasoning to build an explainable ai model for stroke prediction. In: 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE), pp 817–821. IEEE

  • Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A (2018) Catboost: unbiased boosting with categorical features. Adv Neural Inf Proc Syst 31

  • Qing X, Niu Y (2018) Hourly day-ahead solar irradiance prediction using weather forecasts by lstm. Energy 148:461–468. https://doi.org/10.1016/j.energy.2018.01.177

    Article  Google Scholar 

  • Sagi O, Rokach L (2018) Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8(4):1249

  • Schapire RE (2003) The boosting approach to machine learning: An overview. Nonlinear estimation and classification, 149–171

  • Seber GA, Lee AJ (2012) Linear regression analysis. Wiley

  • Sevas MS, Tur Santona CF, Sharmin N (2023) Ensemble machine-learning model for solar radiation prediction using explainable ai. In: 2023 14th International Conference on Computing Communication and Networking Technologies (ICC-CNT), pp 1–6. https://doi.org/10.1109/ICCCNT56998.2023.10307694

  • Sinaga KP, Yang M-S (2020) Unsupervised k-means clustering algorithm. IEEE Access 8:80716–80727

    Article  Google Scholar 

  • Solano ES, Dehghanian P, Affonso CM (2022) Solar radiation forecasting using machine learning and ensemble feature selection. Energies 15(19):7049–7049. https://doi.org/10.3390/en15197049

    Article  Google Scholar 

  • Sushanth K, Mishra A, Mukhopadhyay P, Singh R (2023) Near-real-time forecast- ing of reservoir inflows using explainable machine learning and short-term weather forecasts. Stochastic Environmental Research and Risk Assessment 1–21. https://doi.org/10.1007/s00477-023-02489-y

  • Wang H, Li G, Tsai C-L (2007) Regression coefficient and autoregressive order shrinkage and selection via the lasso. J R Stat Soc Ser B Stat Methodol 69(1):63–78. https://doi.org/10.1111/j.1467-9868,2007.00577.x

    Article  Google Scholar 

  • Wang H, Cai R, Zhou B, Aziz S, Qin B, Voropai N, Gan L, Barakht-enko E (2020) Solar irradiance forecasting based on direct explainable neural network. Energy Convers Manage 226:113487

  • Weber CM, Ray D, Valverde AA, Clark JA, Sharma KS (2022) Gaussian mixture model clustering algorithms for the analysis of high-precision mass mea- surements. Nucl Instrum Methods Phys Res, Sect A 1027:166299

    Article  CAS  Google Scholar 

  • Wu T, Zhang W, Jiao X, Guo W, Hamoud YA (2021) Evaluation of stacking and blending ensemble learning methods for estimating daily reference evapotranspi- ration. Comput Electron Agric 184:106039

    Article  Google Scholar 

  • Zhang Z, Damiani E, Al Hamadi H, Yeun CY, Taher F (2022) Explainable artificial intelligence to detect image spam using convolutional neural network. In: 2022 International Conference on Cyber Resilience (ICCR), pp. 1–5. https://doi.org/10.48550/arXiv.2209.03166. IEEE

Download references

Funding

None.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Military Institute of Science and Technology, Dhaka, Bangladesh

    Muhammad Samee Sevas, Nusrat Sharmin, Chowdhury Farjana Tur Santona & Saidur Rahaman Sagor

Authors
  1. Muhammad Samee Sevas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nusrat Sharmin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chowdhury Farjana Tur Santona
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saidur Rahaman Sagor
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SS did the experiment and wrote the initial draft. NS planned the methodology, revised the manuscript, and supervised the whole work. FS and SRS write the partial experiment and draft.

Corresponding author

Correspondence to Nusrat Sharmin.

Ethics declarations

Competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sevas, M.S., Sharmin, N., Santona, C.F.T. et al. Advanced ensemble machine-learning and explainable ai with hybridized clustering for solar irradiation prediction in Bangladesh. Theor Appl Climatol (2024). https://doi.org/10.1007/s00704-024-04951-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00704-024-04951-5

Search

Navigation