Abstract
This chapter provides a brief and compact overview of the basic terminologies and definitions in Big Data (BD), machine learning (ML), artificial intelligence (AI) and explainable AI (XAI). BD focuses on collecting, cleansing, storing, analyzing, extracting information from, align with interpreting large datasets in science and businesses from healthcare, media, energy systems to defense. AI and ML-based algorithms are widely used in the research to perform tasks that typically require human intelligence, such as driving cars, translating speech, and image recognition. Machine learning algorithms are generally split into four main categories: Supervised Learning, Unsupervised Learning, Reinforcement Learning, and Ensemble Learning. The traditional AI function as a black-box model typically does not provide decision-makers and domain experts with any guidance as to why a particular decision was made. This lack may often lead decision-makers and domain experts to question the results and ultimately reject them because they cannot explain the approach to stakeholders, politicians, and others who want to know why a particular decision was made. XAI provides a more transparent and explanatory approach, thereby rendering the decision more acceptable to people who need to explain why a particular decision was made to others.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sagiroglu, S. and Sinanc, D., 2013, May. Big data: A review. In 2013 international conference on collaboration technologies and systems (CTS) (pp. 42–47). IEEE.
Gandomi, A. and Haider, M., 2015. Beyond the hype: Big data concepts, methods, and analytics. International journal of information management, 35(2), pp. 137–144.
PCA vs LDA vs T-SNE, https://medium.com/analytics-vidhya/pca-vs-lda-vs-t-sne-lets-understand-the-difference-between-them-22fa6b9be9d0, (Accessed Aug. 29, 2021).
Singh, S. and Silakari, S., 2009. Generalized discriminant analysis algorithm for feature reduction in cyber attack detection system. arXiv preprint arXiv:0911.0787.
ML-Classification vs Regression, https://www.geeksforgeeks.org/ml-classification-vs-regression/, (Accessed Aug. 29, 2021).
Montgomery, D.C., Peck, E.A. and Vining, G.G., 2012. Introduction to linear regression analysis (Vol. 821). John Wiley & Sons.
Hall, L.O., Chawla, N. and Bowyer, K.W., 1998, October. Decision tree learning on very large data sets. In SMC’98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No. 98CH36218) (Vol. 3, pp. 2579–2584). IEEE.
Friedman, J., Hastie, T. and Tibshirani, R., 2001. The elements of statistical learning (Vol. 1, No. 10). New York: Springer series in statistics.
Cali, Umit, and Claudio Lima. “Energy informatics using the distributed ledger technology and advanced data analytics.” Cases on Green Energy and Sustainable Development. IGI Global, 2020. 438–481. (2)
Rahman, I., Kuzlu, M. and Rahman, S., 2018. Power disaggregation of combined HVAC loads using supervised machine learning algorithms. Energy and Buildings, 172, pp. 57–66.
https://analyticsindiamag.com/most-popular-clustering-algorithms-used-in-machine-learning/, (Accessed Aug. 29, 2021).
https://towardsdatascience.com/gaussian-mixture-models-d13a5e915c8e, (Accessed Aug. 29, 2021).
https://www.geeksforgeeks.org/ml-mean-shift-clustering/, (Accessed Aug. 29, 2021).
https://www.analyticsvidhya.com/blog/2016/11/an-introduction-to-clustering-and-different-methods-of-clustering/, (Accessed Aug. 29, 2021).
https://www.freecodecamp.org/news/8-clustering-algorithms-in-machine-learning-that-all-data-scientists-should-know/, (Accessed Aug. 29, 2021).
Sutton, R.S. and Barto, A.G., 2018. Reinforcement learning: An introduction. MIT press.
https://towardsdatascience.com/a-beginners-guide-to-q-learning-c3e2a30a653c, (Accessed Aug. 29, 2021).
https://medium.com/swlh/introduction-to-reinforcement-learning-coding-sarsa-part-4-2d64d6e37617, (Accessed Aug. 29, 2021).
https://towardsdatascience.com/boosting-algorithms-explained-d38f56ef3f30, (Accessed Aug. 29, 2021).
https://www.mygreatlearning.com/blog/bagging-boosting, (Accessed Aug. 29, 2021).
https://towardsdatascience.com/ensemble-methods-bagging-boosting-and-stacking-c9214a10a205, (Accessed Aug. 29, 2021).
Mellit A, Kalogirou SA. Artificial intelligence techniques for photovoltaic applications: A review. Progress in energy and combustion science. 2008 Oct 1;34(5):574–632.
Raza MQ, Khosravi A. A review on artificial intelligence based load demand forecasting techniques for smart grid and buildings. Renewable and Sustainable Energy Reviews. 2015 Oct 1;50:1352–72.
Zhou H, Rao M, Chuang KT. Artificial intelligence approach to energy management and control in the HVAC process: an evaluation, development and discussion. Developments in Chemical Engineering and Mineral Processing. 1993;1(1):42–51.
De Benedetti M, Leonardi F, Messina F, Santoro C, Vasilakos A. Anomaly detection and predictive maintenance for photovoltaic systems. Neurocomputing. 2018 Oct 8;310:59–68.
Utility Dive, How does AI improve grid performance? No one fully understands and that’s limiting its use, [Online]. Available: https://www.utilitydive.com/news/how-does-ai-improve-grid-performance-no-one-fully-understands-and-thats-l/566997/
Ribeiro MT, Singh S, Guestrin C. “Why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining 2016 Aug 13 (pp. 1135–1144).
Shrikumar, A., Greenside, P. and Kundaje, A., 2017. Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685.
Skater: Python library for model interpretation/explanations. GitHub repository, https://github.com/oracle/Skater
Siddhartha M, Maity P, Nath R. Explanatory Artificial Intelligence (Xai) In The Prediction Of Post-Operative Life Expectancy In Lung Cancer Patients. International Journal of Scientific Research. 2020 Jan 24;8(12).
“ELI5.” https://github.com/TeamHG-Memex/eli5, 2019. [Online; Accessed 2019-06-04].
Arya, V., Bellamy, R.K., Chen, P.Y., Dhurandhar, A., Hind, M., Hoffman, S.C., Houde, S., Liao, Q.V., Luss, R., Mojsilovic, A. and Mourad, S., 2020. Ai explainability 360: An extensible toolkit for understanding data and machine learning models. Journal of Machine Learning Research, 21(130), pp. 1–6.
Raschka S. MLxtend: Providing machine learning and data science utilities and extensions to Python’s scientific computing stack. Journal of open source software. 2018 Apr 22;3(24):638.
InterpretML, Explain Your Model with Microsoft’s InterpretML, https://medium.com/analytics-vidhya/explain-your-model-with-microsofts-interpretml-5daab1d693b4
Rulex Explainable AI (XAI), https://www.rulex.ai/rulex-explainable-ai-xai/
FĂ©lix Revert, Interpreting Random Forest and other black box models like XGBoost, https://towardsdatascience.com/interpreting-random-forest-and-other-black-box-models-like-xgboost-80f9cc4a3c38
Seldon, Alibi, https://docs.seldon.io/projects/alibi/en/latest/overview/getting_started.html.
IBM, Contrastive Explanation Method (CEM), https://github.com/IBM/Contrastive-Explanation-Method
Machine Learning Interpretability (MLI), https://github.com/h2oai/mli-resources
XAI – The eXplainable AI Framework, https://ethical.institute/xai.html
Joshua Poduska, SHAP and LIME Python Libraries: Part 1 – Great Explainers, with Pros and Cons to Both, https://blog.dominodatalab.com/shap-lime-python-libraries-part-1-great-explainers-pros-cons/
Sumit Saha, local interpretable model-agnostic explanations (lime) – the eli5 way, https://medium.com/intel-student-ambassadors/local-interpretable-model-agnostic-explanations-lime-the-eli5-way-b4fd61363a5e
Marco Tulio Ribeiro, LIME – Local Interpretable Model-Agnostic Explanations, https://homes.cs.washington.edu/~marcotcr/blog/lime/
Sumit Saha, local interpretable model-agnostic explanations (lime) – the eli5 way, https://medium.com/intel-student-ambassadors/local-interpretable-model-agnostic-explanations-lime-the-eli5-way-b4fd61363a5e
Eriksson, T., 2020. Occlusion method to obtain saliency maps for CNN.
Skater Overview, https://oracle.github.io/Skater/overview.html
Skater, https://github.com/oracle/Skater
Hart, S., 1989. Shapley value. In Game Theory (pp. 210–216). Palgrave Macmillan, London.
Lundberg, Scott M., and Su-In Lee. “A unified approach to interpreting model predictions.” Advances in Neural Information Processing Systems. 2017
towardsdatascience, Explain Your Model with the SHAP Values, https://towardsdatascience.com/explain-your-model-with-the-shap-values-bc36aac4de3d
On Click 260, nterpretable Machine Learning With Lime+ELI5+SHAP+InterpretML, https://www.onclick360.com/interpretable-machine-learning-with-lime-eli5-shap-interpret-ml/
ELI5, ELI5, https://github.com/TeamHG-Memex/eli5
AI Explainability 360, https://aix360.readthedocs.io/en/latest/
Raschka, S., 2018. MLxtend: providing machine learning and data science utilities and extensions to Python’s scientific computing stack. Journal of open source software, 3(24), p.638.
Nori, H., Jenkins, S., Koch, P. and Caruana, R., 1909. InterpretML: A Unified Framework for Machine Learning Interpretability (2019). arXiv preprint arXiv:1909.09223.
InterpretML - Alpha Release, https://github.com/interpretml/interpret
Alibi Explain, https://github.com/SeldonIO/alibi
Dhurandhar, A., Chen, P.Y., Luss, R., Tu, C.C., Ting, P., Shanmugam, K. and Das, P., 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems (pp. 592–603).
Explainable AI: The Next Best Thing in Digital Health, https://ekare.ai/explainable-ai-the-next-best-thing-in-digital-health/
An Introduction to eXplainable AI with H2O Driverless AI, https://andisama.medium.com/an-introduction-to-explainable-ai-with-h2o-driverless-ai-2a9e8f27e03f
XAI – An eXplainability toolbox for machine learning, https://github.com/EthicalML/xai
Carvalho, D.V., Pereira, E.M. and Cardoso, J.S., 2019. Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8), p. 832.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Cali, U., Kuzlu, M., Pipattanasomporn, M., Kempf, J., Bai, L. (2021). Foundations of Big Data, Machine Learning, and Artificial Intelligence and Explainable Artificial Intelligence. In: Digitalization of Power Markets and Systems Using Energy Informatics. Springer, Cham. https://doi.org/10.1007/978-3-030-83301-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-83301-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-83300-8
Online ISBN: 978-3-030-83301-5
eBook Packages: EnergyEnergy (R0)