Abstract
Businesses nowadays often feel impelled to rush and implement Big Data and Artificial Intelligence initiatives in their organizations without clarity on their business problems, nor the appropriate methodologies for extracting actionable insights from the data. In contrast, this paper presents a process that starts with an articulated business problem instead of a “data rush”. The presented process, dubbed the Business-Driven Data-Supported (BDDS) process, is rigorously anchored in concepts from Theory of Constraints and Information Theory. BDDS guides businesses in solving their problems by stating observed performance gaps, uncovering their underlying root cause, formulating the “right question”, utilizing only the “right data”, and choosing the “right methodology” to extract the “right information” from the data, leading to the “right actionable insights.” Also provided is a prescriptive framework, dubbed the Data-to-Information-Extraction-Methodology (DIEM), for selecting the “right methodology”, based on the understanding level of relevant system dependencies and the availability of relevant data. The BDDS process is illustrated by an example from the healthcare industry, and the efficacy and applicability of the DIEM framework are supported by eleven case studies from a broad range of industries.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ackoff, R. (1989). From data to wisdom. Journal of Applied Systems Analysis, 16(1), 3–9.
Akter, S., Michael, K., Uddin, M. R., McCarthy, G., & Rahman, M. (2022). Transforming business using digital innovations: The application of AI, blockchain, cloud and data analytics. Annals of Operations Research, 308(1–2), 7–39. https://doi.org/10.1007/s10479-020-03620-w
Akter, S., Wamba, S. F., Gunasekaran, A., Dubey, R., & Childe, S. J. (2016). How to improve firm performance using big data analytics capability and business strategy alignment? International Journal of Production Economics, 182, 113–131.
Amalina, F., Hashem, I. A. T., Azizul, Z. H., Fong, A. T., Firdaus, A., Imran, M., & Anuar, N. B. (2019). Blending big data analytics: Review on challenges and a recent study. IEEE Access, 8, 3629–3645.
Babu, M. M., Rahman, M., Alam, A., & Dey, B. L. (2021). Exploring big data-driven innovation in the manufacturing sector: Evidence from UK firms. Annals of Operations Research. https://doi.org/10.1007/s10479-021-04077-1
Bag, S. (2017). Big data and predictive analysis is key to superior supply chain performance: A South African experience. International Journal of Information Systems and Supply Chain Management (IJISSCM), 10(2), 66–84.
Belhadi, A., Mani, V., Kamble, S. S., Khan, S. A. R., & Verma, S. (2021). Artificial intelligence-driven innovation for enhancing supply chain resilience and performance under the effect of supply chain dynamism: An empirical investigation. Annals of Operations Research. https://doi.org/10.1007/s10479-021-03956-x
Binning, D. (2016). Atlassian grows through rapid trial and error, rather than analysis. Financial Review.
Breyfogle, F. W., III. (2003). Implementing six sigma: Smarter solutions using statistical methods. Wiley.
Brice, R. (2013). Tesco improves supply chain with big data, automated data collection. https://www.rfgen.com/blog/tesco-improves-supply-chain-with-big-data-automated-data-collection/
Brown, M. S. (2016). Analytics and the IRS: A new way to find cheaters. Forbes. https://www.forbes.com/sites/metabrown/2016/01/28/analytics-and-the-irs-a-new-way-to-find-cheaters/#5a4f93853187
Bumblauskas, D., Nold, H., Bumblauskas, P., & Igou, A. (2017). Big data analytics: Transforming data to action. Business Process Management Journal, 23(3), 703–720.
Carillo, K. D. A. (2017). Let’s stop trying to be “sexy”—preparing managers for the (big) data-driven business era. Business Process Management Journal.
Chatterjee, S., Chaudhuri, R., & Demetris-Vrontis, D. (2021). Does data-driven culture impact innovation and performance of a firm? An empirical examination. Annals of Operations Research.
Chen, J., Lim, C. P., Tan, K. H., Govindan, K., Kumar, A. (2021). Artificial intelligence-based human-centric decision support framework: An application to predictive maintenance in asset management under pandemic environments. Annals of Operations Research.
Choi, T., Wallace, S. W., & Wang, Y. (2018). Big data analytics in operations management. Production and Operations Management, 27(10), 1868–1883.
Chui, M., & Fleming, T. (2011). The benefits—and limits—of decision models. McKinsey Quarterly, November. http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Inside+P+&+G+’+s+digital+revolution#0
Clancy, R., O’Sullivan, D., & Bruton, K. (2023). Data-driven quality improvement approach to reducing waste in manufacturing. The TQM Journal, 35(1), 51–72.
Court, D., & Barton, D. (2013). Making advanced analytics work for you. Harvard Business Review, pp. 1–9.
Court, D., Jesko, P., McGuire, T., Gordon, J., & Spillecke, D. (2015). Marketing & sales–big data, analytics, and the future of marketing & sales.
Dash, S., Shakyawar, S. K., Sharma, M., & Kaushik, S. (2019). Big data in healthcare: Management, analysis and future prospects. Journal of Big Data, 6(1), 1–25.
Davenport, T., & Harris, J. (2017). Competing on analytics: Updated, with a new introduction: The new science of winning. Harvard Business Press.
DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60–95.
Dhaenens, C., & Jourdan, L. (2022). Metaheuristics for data mining: Survey and opportunities for big data. Annals of Operations Research, 314(1), 117–140.
Dykes, B. (2016). Actionable insights: The missing link between data and business value. https://www.forbes.com/sites/brentdykes/2016/04/26/actionable-insights-the-missing-link-between-data-and-business-value/#16d3a27151e5
General Electric (2023). Project predictive maintenance. https://www.ge.com/research/project/predictive-maintenance
Eliot, T. S. (2011). The complete poems and plays of TS Eliot. Faber & Faber.
Fidelman, M. (2012). Why Atlassian is to software as apple is to design. Forbes.
Forbes (2017). How AI and machine learning are helping drive the GE Digital transformation. Forbes. https://www.forbes.com/sites/ciocentral/2017/06/07/how-ai-and-machine-learning-are-helping-drive-the-ge-digital-transformation/?sh=475287ce1686
Fung, K. (2014). Google Flu Trends’ failure shows good data > big data. Harvard Business Review, MARCH 25. https://hbr.org/2014/03/google-flu-trends-failure-shows-good-data-big-data
Gallo, C. (2011). Steve jobs: Get rid of the crappy stuff. Forbes.
Garajedaghi, J. (1999). Systems thinking: Managing chaos and complexity. Butterworth-Heinemann Boston.
Ghallab, M., Nau, D., & Traverso, P. (2004). Automated planning: Theory and practice. Elsevier.
Goldratt E. M., Cox J. (2014). The goal: A process of ongoing improvement–30th anniversary edition. 4. rev. ed., 30th anniversary ed. Great Barrington: North River Press.
Goldratt, E. M. (2017). Critical chain: A business novel. Routledge.
Goldratt, E. M. (1990a). What is this thing called theory of constraints and how should it be implemented? North River Press.
Goldratt, E. M. (1990b). The haystack syndrome: Sifting information out of the data. North River Press.
Goldratt, E. M. (1999). Theory of constraints. North River Press.
Gozalo-Brizuela, R., & Garrido-Merchan, E. C. (2023). ChatGPT is not all you need. A state of the art review of large generative AI models. arXiv:2301.04655.
Grover, P., Kar, A. K., & Dwivedi, Y. K. (2022). Understanding artificial intelligence adoption in operations management: Insights from the review of academic literature and social media discussions. Annals of Operations Research, 308(1–2), 177–213.
Gupta, S., Modgil, S., Bhattacharyya, S., & Bose, I. (2022). Artificial intelligence for decision support systems in the field of operations research: Review and future scope of research. Annals of Operations Research.
Gupta, S., Altay, N., & Luo, Z. (2019). Big data in humanitarian supply chain management: A review and further research directions. Annals of Operations Research, 283, 1153–1173.
Hastie, T., Tibshirani, R., Friedman, J. H., & Friedman, J. H. (2009). The elements of statistical learning: data mining, inference, and prediction (Vol. 2, pp. 1–758). New York: Springer.
Hayes, J. L., Britt, B. C., Evans, W., Rush, S. W., Towery, N. A., & Adamson, A. C. (2021). Can social media listening platforms’ artificial intelligence be trusted? Examining the accuracy of Crimson Hexagon’s (now Brandwatch Consumer Research’s) AI-Driven analyses. Journal of Advertising, 50(1), 81–91.
Holland, C., Levis, J., Nuggehalli, R., Santilli, B., & Winters, J. (2017). UPS optimizes delivery routes. Interfaces, 47(1), 8–23. https://doi.org/10.1287/inte.2016.0875
Houser, K. A., & Sanders, D. (2017). The use of big data analytics by the IRS: Efficient solutions or the end of privacy as we know it? Vanderbilt Journal of Entertainment & Technology Law, 19(4), 817–872.
James, J. T. (2013). A new, evidence-based estimate of patient harms associated with hospital care. Journal of Patient Safety, 9(3), 122–128.
Johnson, M. (2017). Starbucks’ digital flywheel program will use artificial intelligence. https://www.zacks.com/stock/news/270022/starbucks-digital-flywheel-program-will-use-artificial-intelligence
Josephs, L. (2018). Here’s why GE is holding on tight to its aviation business. CNBC. https://www.cnbc.com/2018/06/26/why-general-electric-is-holding-on-tight-to-its-aviation-business.html
Juran, J. M., Gryna, F. M., & Bingham, R. S. (1974). Quality control handbook (Vol. 3). McGraw-Hill.
Kamble, S. S., Gunasekaran, A., & Gawankar, S. A. (2020). Achieving sustainable performance in a data-driven agriculture supply chain: A review for research and applications. International Journal of Production Economics, 219, 179–194.
Kaplan, A., & Haenlein, M. (2019). Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons, 62(1), 15–25. https://doi.org/10.1016/j.bushor.2018.08.004
Karakusevic, S. (2016). Understanding patient flow in hospitals. Nuffield Trust.
Kash, D. E., & Rycroft, R. W. (2003). To manage complex innovation, ask the right questions. Research-Technology Management, 46(5), 29–33. https://doi.org/10.1080/08956308.2003.11671585
Knight, A. (2014). Pride and Joy. Church Farm: Never Say I Know.
Kohavi, R., & Thomke, S. (2017). The surprising power of online experiments. Harvard Business Review.
Konrad, A. (2013). Meet ORION, Software That Will Save UPS Millions By Improving Drivers’ Routes. Forbes. https://www.forbes.com/sites/alexkonrad/2013/11/01/meet-orion-software-that-will-save-ups-millions-by-improving-drivers-routes/?sh=dd14ae4fc713
Kumar, C. (2018). Artificial intelligence: Definition, types, examples, technologies. https://chethankumargn.medium.com/artificial-intelligence-definition-types-examples-technologies-962ea75c7b9b
LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S., & Kruschwitz, N. (2011). Big data, analytics and the path from insights to value. MIT Sloan Management Review, 52(2), 21–32.
Freeman Law. (2018). IRS use of big data leads to 400% increase in detection of tax fraud by criminal investigation division. Freeman Law. https://www.freemanlaw-pllc.com/irs-use-of-big-data-leads-to-400-increase-in-detection-of-tax-fraud-by-criminal-investigation-division/
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). Supplementary materials for the parable of google flu: Traps in big data analysis. Science, 343, 1203–1206. https://doi.org/10.1126/science.1248506
Li, L., Chi, T., Hao, T., & Yu, T. (2018). Customer demand analysis of the electronic commerce supply chain using big data. Annals of Operations Research, 268(1–2), 113–128. https://doi.org/10.1007/s10479-016-2342-x
Lombardi, A. M. (2017). The epistemic and aleatory uncertainties of the ETAS-type models: An application to the Central Italy seismicity. Scientific Reports, 7(1), 1–9.
Lopez, E. (2017). What is big data, and why does it matter to supply chain? Supply Chain Dive.
Loshin, D. (2010). Master data management. Morgan Kaufmann.
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., & Byers, A. H. (2011). Big data: The next frontier for innovation, competition, and productivity. McKinsey Global Institute.
Marr, B. (2016). Big data at Tesco: Real time analytics at the UK Grocery Retail Giant. Forbes. https://www.forbes.com/sites/bernardmarr/2016/11/17/big-data-at-tesco-real-time-analytics-at-the-uk-grocery-retail-giant/#4121c01b61cf
Marr, B. (2017). The amazing Way GE is combining big data and electrons to create “The Internet of Energy.” Forbes. https://www.forbes.com/sites/bernardmarr/2017/03/28/the-amazing-way-ge-is-combining-big-data-and-electrons-to-create-the-internet-of-energy/#5219949f1806
Marr, B. (2018a). Starbucks: Using big data, analytics and artificial intelligence to boost performance. Forbes.
Marr, B. (2018b). The Brilliant ways UPS uses artificial intelligence, machine learning and big data. Forbes.
Mascarenhas, B., Baveja, A., & Jamil, M. (1998). Dynamics of core competencies in leading multinational companies. California Management Review, 40(4), 117–132. https://doi.org/10.2307/41165967
Mauri, A. G., Minazzi, R., & Muccio, S. (2013). A review of literature on the gaps model on service quality: A 3-decades period: 1985–2013. International Business Research, 6(12), 134.
McAfee, A., Brynjolfsson, E., Davenport, T. H., Patil, D. J., & Barton, D. (2012). Big data: The management revolution. Harvard Business Review, 90(10), 60–68.
McCain, S. L. C., Jang, S. S., & Hu, C. (2005). Service quality gap analysis toward customer loyalty: Practical guidelines for casino hotels. International Journal of Hospitality Management, 24(3), 465–472.
Meadows, D. H. (2008). Thinking in systems: A primer. Chelsea Green Publishing.
Miller, J. W., Ganster, D. C., & Griffis, S. E. (2018). Leveraging big data to develop supply chain management theory: The case of panel data. Journal of Business Logistics, 39(3), 182–202.
Moore, S. M., Diener, E., & Tan, K. (2018). Using multiple methods to more fully understand causal relations: Positive affect enhances social relationships. DEF Publishers.
Nagendra, N. P., Narayanamurthy, G., & Moser, R. (2020). Management of humanitarian relief operations using satellite big data analytics: The case of Kerala floods. Annals of Operations Research, pp. 1–26.
NewVantage Partners LLC. (2017). Big data executive survey 2017—Executive summary of findings
Niederman, F., Brancheau, J. C., & Wetherbe, J. C. (1991). Information systems management issues for the 1990s. MIS Quarterly, 15(4), 475–500.
Okręglicka, M., Gorzeń-Mitka, I., & Ogrean, C. (2015). Management challenges in the context of a complex view—SMEs perspective. Procedia Economics and Finance, 34, 445–452. https://doi.org/10.1016/S2212-5671(15)01653-6
Olsen, L., Saunders, R. S., & Yong, P. L. (2010). The healthcare imperative: Lowering costs and improving outcomes: Workshop series summary.
Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1985). A conceptual model of service quality and its implications for future research. Journal of Marketing, 49(4), 41–50.
Prahalad, C. K., & Hamel, G. (1990). The core competence of the corporation. Harvard Business Review, May–June.
Provost, F., & Fawcett, T. (2013). Data Science for Business: What you need to know about data mining and data-analytic thinking. O'Reilly Media, Inc.
Raguseo, E., Vitari, C., & Pigni, F. (2020). Profiting from big data analytics: The moderating roles of industry concentration and firm size. International Journal of Production Economics, 229, 107758.
Rappeport, A. (2021). Tax cheats cost the U.S. $1 trillion per year, I.R.S. chief says. The New York Times. https://www.nytimes.com/2021/04/13/business/irs-tax-gap.html
Reichheld, F. (2015). Seeing through the big data hype. Bain & Company.
Rialti, R., Marzi, G., Silic, M., & Ciappei, C. (2018). Ambidextrous organization and agility in big data era: The role of business process management systems. Business Process Management Journal.
Robins, S. P., Decenzo, D. A., & Coulter, M. (2013). Fundamentals of Management Essential Concepts and Applications (Eighth). Pearson.
Roe, J., & Elton, J. (1998). Bringing discipline to project management. Harvard Business Review, 76(2), 153–160.
Rowley, J. (2007). The wisdom hierarchy: Representations of the DIKW hierarchy. Journal of Information Science, 33(2), 163–180. https://doi.org/10.1177/0165551506070706
Russell, S. J., & Norvig, P. (2016). Artificial intelligence: a modern approach (3rd ed.). Pearson Education Limited.
Sagiroglu, S., & Sinanc, D. (2013). Big data: A review. International Conference on Collaboration Technologies and Systems (CTS), 2013, 42–47. https://doi.org/10.1109/CTS.2013.6567202
Sanders, N. R. (2016). How to use big data to drive your supply chain. California Management Review, 58(3), 26–48. https://doi.org/10.1525/cmr.2016.58.3.26
SAS. (2022). Big data what it is and why it matters. https://www.sas.com/en_in/insights/big-data/what-is-big-data.html#:~:text=Big%20data%20is%20a%20term,with%20the%20data%20that%20matters.
See-To, E. W., & Ngai, E. W. (2018). Customer reviews for demand distribution and sales nowcasting: A big data approach. Annals of Operations Research, 270, 415–431.
Seth, N., Deshmukh, S. G., & Vrat, P. (2006). A framework for measurement of quality of service in supply chains. Supply Chain Management: An International Journal, 11(1), 82–94.
Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication (p. 97). University of Illinois Press.
SiSense. (2018). Business analytics and the data complexity matrix. https://www.sisense.com/blog/know-your-data-the-data-complexity-matrix/
Smith, G. (2020). Data mining fool’s gold. Journal of Information Technology, 35(3), 182–194.
Srinivasan, M. M., Best, W. D., & Chandrasekaran, S. (2007). Warner Robins Air Logistics Center streamlines aircraft repair and overhaul. Interfaces, 37(1), 7–21. https://doi.org/10.1287/inte.1060.0260
Starbucks. (2016). 10 Big announcements from Starbucks 2016 Investor Day.
Starbucks. (2017). Fortune Names Starbucks 2017’s Third Most Admired Company in the World. https://news.starbucks.com/news/fortune-most-admired-company-2017
Starbucks. (2019). Starbucks reports Q1 Fiscal 2019 Results. https://investor.starbucks.com/press-releases/financial-releases/press-release-details/2019/Starbucks-Reports-Q1-Fiscal-2019-Results/default.aspx.
Statista Research Department. (2021). Big data market size revenue forecast worldwide from 2011 to 2027. https://www.statista.com/statistics/254266/global-big-data-market-forecast/
Stephenson, D. (2013). 7 Big data techniques that create business value. Touchpoint by Firmex. https://www.firmex.com/resources/blog/7-big-data-techniques-that-create-business-value/
Stobierski, T. (2019). The advantages of data-driven decision making. Harvard Business School Online. https://online.hbs.edu/blog/post/data-driven-decision-making#:~:text=Benefits%20of%20Data-Driven%20Decision-Making%201%201.%20You%E2%80%99ll%20Make,...%203%203.%20You%20Can%20Realize%20Cost%20Savings
The Guardian. (2014). Google Flu Trends is no longer good at predicting flu, scientists find. The Guardian. https://www.theguardian.com/technology/2014/mar/27/google-flu-trends-predicting-flu
Taecharungroj, V. (2023). “What Can ChatGPT Do?” Analyzing early reactions to the innovative Ai Chatbot on twitter. Big Data and Cognitive Computing, 7(1), 35.
TechHQ. (2017). How Starbucks uses data and insights to win big. TechHQ.
Tsai, W.-H., Hsu, W., & Chou, W.-C. (2011). A gap analysis model for improving airport service quality. Total Quality Management & Business Excellence, 22(10), 1025–1040.
Tsolakis, N., Schumacher, R., Dora, M., & Kumar, M. (2023). Artificial intelligence and Blockchain implementation in supply chains: A pathway to sustainability and data monetisation? Annals of Operations Research, 327(1), 157–210.
Tufte, E. R. (2001). The visual display of quantitative information (Vol. 2). Cheshire: Graphics Press.
Vasconcelos, F. C., & Ramirez, R. (2011). Complexity in business environments. Journal of Business Research, 64(3), 236–241. https://doi.org/10.1016/j.jbusres.2009.11.007
Verbeke, W., Baesens, B., & Bravo, C. (2017). Profit driven business analytics: A practitioner’s guide to transforming big data into added value. Wiley
Voosen, P. (2017). How AI detectives are cracking open the black box of deep learning. Science. http://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning
Waller, M. A., & Fawcett, S. E. (2013a). Click here for a data scientist: Big data, predictive analytics, and theory development in the era of a maker movement supply chain. Journal of Business Logistics, 34(4), 249–252.
Waller, M. A., & Fawcett, S. E. (2013b). Data science, predictive analytics, and big data: A revolution that will transform supply chain design and management. Journal of Business Logistics, 34(2), 77–84.
Wang, G., Gunasekaran, A., Ngai, E. W. T., & Papadopoulos, T. (2016). Big data analytics in logistics and supply chain management: Certain investigations for research and applications. International Journal of Production Economics, 176, 98–110. https://doi.org/10.1016/j.ijpe.2016.03.014
Wang, R. Y., & Strong, D. M. (1996). Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems, 12(4), 5–33.
Winch, G., Usmani, A., & Edkins, A. (1998). Towards total project quality: A gap analysis approach. Construction Management & Economics, 16(2), 193–207.
Winnig, L. (2016). GE’s big bet on data and analytics. MIT Sloan Management Review, 57(3), 1–22.
Winslow, B. (2018). Utah highlights reform efforts after lawsuit over mentally ill inmates being stuck in jails indefinitely. Fox 13. https://fox13now.com/2018/07/19/utah-highlights-reform-efforts-after-lawsuit-over-mentally-ill-inmates-being-stuck-in-jails-indefinitely/
Wischmeyer, T. (2020). Artificial intelligence and transparency: opening the black box. Regulating Artificial Intelligence, pp. 75–101.
Yoshida, S., Matsushima, M., Wakabayashi, H., Mutai, R., Murayama, S., Hayashi, T., Ichikawa, H., Nakano, Y., Watanabe, T., & Fujinuma, Y. (2017). Validity and reliability of the Patient Centered Assessment Method for patient complexity and relationship with hospital length of stay: A prospective cohort study. British Medical Journal Open, 7(5), e016175.
Yu, Y. (2017). Starbucks–grinding data and brewing results. https://digit.hbs.org/submission/starbucks-grinding-data-and-brewing-results/
Yu, W., Wong, C. Y., Chavez, R., & Jacobs, M. A. (2021). Integrating big data analytics into supply chain finance: The roles of information processing and data-driven culture. International Journal of Production Economics, 236, 108135.
Zhan, Y., Tan, K. H., Li, Y., & Tse, Y. K. (2018). Unlocking the power of big data in new product development. Annals of Operations Research, 270(1–2), 577–595. https://doi.org/10.1007/s10479-016-2379-x
Zhu, J. (2022). DEA under big data: Data enabled analytics and network data envelopment analysis. Annals of Operations Research, 309(2), 761–783.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rodgers, M., Mukherjee, S., Melamed, B. et al. Solving business problems: the business-driven data-supported process. Ann Oper Res 332, 705–741 (2024). https://doi.org/10.1007/s10479-023-05770-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-023-05770-z