Introduction

Big data and predictive analytics (BDPA) are used for several purposes, such as optimisation, planning, strategizing, and resource control, with the overall goal of improved performance. BDPA also facilitate links between businesses and customers. By drawing on analytics of preferences and experiences, managers can improve customer experience and manage their products and processes according to the inclinations of consumers. Despite the growing interest in big data and predictive analytics (BDPA) technology, there is still a research gap in its applications and adoption among micro-, small, and medium enterprises (MSMEs) in Malaysia. While some studies have examined the use of BDPA among larger enterprises, research specifically focussed on BDPA practices in MSMEs is limited. Another gap in the research is the limited attention given to the barriers and challenges faced by MSMEs in adopting BDPA. While BDPA has the potential to enhance the competitiveness of MSMEs in the global market, there may be financial, technical, and human resource-related challenges that need to be addressed before MSMEs can fully embrace this technology. Moreover, there is a lack of research examining the relationship between BDPA and innovation in MSMEs in Malaysia. BDPA has the potential to drive innovation in MSMEs by providing valuable insights into customer behaviour, market trends, and product demand. Therefore, a deeper understanding of the relationship between BDPA and innovation in MSMEs could be beneficial for policymakers and business owners alike.

Contemporary businesses adopt a wide range of innovations in order to gain a competitive edge over their counterparts. BDPA is an emerging field of innovation. The term “big data” refers to data that are complex, large in volume, and accumulated with high velocity. Because of these characteristics, big data require specific storage, management, and analytical technologies and approaches (Chen et al. 2012; McAfee et al. 2012). Predictive analytics belong to the field of information technology and include statistical and empirical methods that generate predictions. In addition, predictive analytics assess prediction quality (Schmueli and Koppius 2011). “Big data and predictive analytics” is a term that encompasses the handling of big data that inform business analytics and data-driven decision-making (Gunaserakan et al. 2017). A study of 330 North American companies established that those which describe themselves as data driven perform better in key areas, such as finance and operations management, than their non-data-driven counterparts (McAffee et al. 2012).

BDPA applications

BDPA have a wide range of applications in, among others, market intelligence (Xu et al. 2016), retail (Aloysius et al. 2018), supply chain management (Gunaserakan et al. 2017), governance (Desouza and Jacob 2017), and healthcare (Harerimana et al. 2018). Wang et al. (2016b, a) asserted that there is a growing need for businesses to understand the role of different types of big data analysis as strategic assets. According to these authors, it is important to understand the ways in which big data and supply chain analytics can be integrated in order to harness their potential. The application of big data analytics (BDA) enhances organisational productivity and innovation diffusion (Ciampi et al. 2021; Zheng et al. 2020) Moreover, BDPA could be utilised by MSMEs for quality evaluation and management (Sariyer et al. 2021).

The perceived benefits of predictive analytics include superior executive capabilities, improved supply chain efficiency, enhanced capabilities to plan for fluctuations in demand, and higher visibility (Schoenherr and Speier‐Pero 2015; Khan et al. 2021). Gunaserakan et al. (2017) established that, if conceptualised as a capability, BDPA positively impact supply chain management and organisational performance. Green data analytics and supply chain management green practices possess a significant association with significant positive effect on three organisational performance dimensions (i.e. operational, environmental and economic performance) (Khan et al. 2021). Big Data help both developing and developed countries to prosper in e-commerce as they improve the company’s growth prospects in different domains, such as marketing, human resource management (HRM), and finance. While we acknowledge the impact of BDPA impacts on firm performance, more recent studies capture these effects on business performance in a more actional and practical manner such as quantifying the adoption of new products innovations and how BDPA facilitate such performance (Duan et al. 2020; Mikalef 2019). More specifically, the number of patents, radical and incremental innovation capabilities and perceived innovation performance compared to competitors. In other word, this is considered as one of the perceived benefits from BDPA adoption. Many companies could track behaviours and retain large numbers of current and potential customers by analysing big data (Akter and Wamba 2016).

BDPA and micro-, small and medium enterprises

Previous studies focussed largely on discussing the influence of BDPA on large established organisations (Akter, et al. 2020a, b; Bertello et al. 2020), while MSMEs have attracted limited scholarly attention. The exploitation of BDPA and supply chain analytics has propelled U.S. e-commerce revenues, currently at 431.6 billion U.S. dollars and projected to increase to 563 billion U.S. dollars by 2025, to a leading position globally (Statista 2021). Newly industrialised countries, which are often in the limelight due to their rapid growth and digitalisation, are also benefiting from the use of BDPA. For instance, in Indonesia, BDPA was utilised to maximise profits by identifying and analysing consumer profiles that were then used as inputs in product innovation (Anugerah and Indriani 2018). In Singapore, BDPA aided Y ventures in identifying trends across more than 28 online markets, including Amazon and Q0010 (Weizen 2017). Since declaring independence in 1957, Malaysia has been diversifying its agriculture-and-commodities-based economy while expanding manufacturing and services. Since 2010, the Malaysian economy has been growing steadily, at an annual average rate of 5.4%, and it is expected to complete its transition from an upper-middle-income to a high-income economy by 2024 (World Bank 2021). MSMEs are considered at ‘the heart’ of the global business economy (Weaven et al. 2021; OECD 2019) with 99% of all firms (Liu et al. 2020). MSMEs are an important part of Malaysia’s development and industrialisation. For instance, the contribution of small and medium enterprises to GDP was 38.2% in 2020 (Department of Statistics 2020).

Although the adoption of BDPA in the Malaysian market is at its infant stage, the National Big Data Analytics Framework has catalysed their integration across sectors. This process has included the integration of BDPA into the operations of MSMEs, which has fostered economic growth and cross-border expansion (National E-Commerce Roadmap 2019). Recent statistics show that around 80% of the Malaysian population are active Internet users and that 62% are using mobile devices for online shopping (International Trade Administration, 2020). MSMEs could benefit from investing in BDPA as it may increase productivity levels and customer responsiveness, which, in turn, is likely to improve management practices (Akarsha 2017). Therefore, main aim of the Malaysian Digital Economy Corporation is to integrate BDPA into all business sectors so as to induce gains in workplace efficiency and innovation (Azman et al. 2021).

This paper examines the relationship between the wider business environment and key firm internal factors that are related to the adoption of BDPA. To be more precise, we first explore the rate of BDPA diffusion among micro-, small, and medium enterprises (MSMEs) in several Malaysian industries. We approached our research with two main research questions in mind, which enabled us to structure our three explorative enquiries:

  1. (1)

    What is the rate of BDPA adoption among Malaysian MSMEs?

  2. (2)

    What are the most prominent applications of BDPA?

  3. (3)

    What factors serve as barriers and facilitators of BDPA adoption among Malaysian MSMEs?

We answered these questions by means of a questionnaire that we distributed to a network of small businesses. We asked our respondents to identify the one predominant use of BDPA that they had found useful in their day-to day operations. These uses include supplier sourcing, customer relationship management, operations and process monitoring, improvement, and product optimisation (Sodero et al. 2019). Our research is grounded in the innovation diffusion theory (IDT; Rogers 2003) and technology–organisation–environment (TOE) framework (Tornarzky and Fleisher 1990). Our theoretical foundation allows us to achieve the two main research goals that we identified. First, we explore the adoption of BDPA among MSMEs from different sectors. This will later allow us and other researchers to develop sophisticated comparative analyses of the use of BDPA in different industries. Second, the theoretical framework allows us to inquire into the factors that facilitate or obstruct the diffusion of innovations, such as BDPA.

We make several contributions to the field of MSME analytics and operations. First, we integrate the TOE framework into IDT. The latter serves to inform the environmental aspects of the former because innovations are diffused in the wider business environment. Thus, sometimes, they can reach critical mass or create competitive pressure. Our theoretical framework informs enquiry about the adoption of BDPA, and we confirm the antecedents of BDPA adoption. We explore the existence of BDPA practices among Malaysian MSMEs and the principal applications of BDPA. Finally, we compare enterprises of different calibres. According to our results, the chief beneficiaries of big data are microbusinesses, followed by small and medium-sized ones. BDPA are most commonly used for customer relationship management (CRM) and for warehouse and operations management. In our analysis, we found a strong link between BDPA and competitive advantage, which evinces the benefits of this technology. Still, less than 20% of medium-sized enterprises use BDPA, while around 33% of small businesses benefit from it. It appears that awareness of the benefits of BDPA must be raised. This process could be facilitated by appropriate policy measures that aim to protect personal information while encouraging MSMEs to utilise relevant data and to improve their performance and outputs.

The remainder of the paper is organised as follows: the first theme considered within the extant literature is the use of BDPA in operation performance including technology diffusion. Moving to the following theme which we build the scope of our hypotheses development on, the application of technology–organisation–environment framework to identify, understand and seize business opportunities to ultimately gains competitive advantage. This requires a close alignment of MSMEs to adjust and adapt the business operations accordingly (Hazen et al. 2014; Osiyevskyy et al. 2020; Weaven et al. 2021). Recent articles highlighted the capabilities of BDPA allowing MSMEs to grasp changes in markets and exploit potential opportunities (Bertello et al. 2020). This is reflected on the context of our study, that is, the steady growth of newly industrialised Malaysia. Next, we elaborate on our theoretical framework, and we abstract our hypotheses. This section is followed by an explanation of our research design and methodology. Thereafter, we present our analysis, and we discuss our findings. Finally, we present the limitations of the paper and avenues for future research.

Theoretical background and hypotheses development

BDPA adoption and MSEMs performance: a innovation diffusion theory and TOE technology–organisation–environment framework view

Our study relies on insights from Innovation Diffusion Theory IDT (Rogers 2003) and the technology–organisation–environment TOE framework (Tornarzky and Fleisher 1990). An innovation is “an idea, practice, or object that is perceived as new by an individual or other unit of adoption” (Rogers 2003, p. 11). IDT has been applied in various empirical studies, for instance, in the context of electronic data interchanges (Premkumar et al. 1994), green innovation (Horbach 2008), and promotional campaigns (Hu et al. 2018). The TOE framework was created by Tornatzky and Fleisher (1990). Initially, it was applied to the field of engineering and technology. TOE describes the factors that influence the adoption of technology and its likelihood. TOE postulates that the adoption and implementation of technological innovations is bounded by a three-dimensional context, which is technological, organisational and environmental (Tornatzky and Fleisher 1990).

The technological context includes internal and external technologies that are relevant to the organisation, such as equipment, know-how, and processes. The organisational context refers to the characteristics and resources of the firm, including its size, its degree of centralisation and formalisation, its managerial structure, its human resources, and linkages between employees. The environmental context subsumes industry size and structure, the firm’s competitors, the macroeconomic context, and the regulatory environment (Tornatzky and Fleisher 1990).

These three elements—technology, environment, and organisation—create “both constraints and opportunities for technological innovation” (Tornatzky and Fleisher 1990, p. 154). Multiple theories have been used to examine the diffusion of technology among MSMEs. Grandon and Pearson (2004) used the technology acceptance model (TAM), while Venkatesh, (2000) applied TAM2. Mehrtens et al. (2001) utilised the resource-based view and the diffusion of innovation (DOI) theory. Developed in 1990 by Tornatzky and Fleischer, the TOE is an integrative framework which provides a theoretical basis for the factors that facilitate or inhibit the diffusion of technology. It has been adopted in e-business (Zhu et al. 2006) and in the field of electronic data interchanges (Kuan and Chau 2001). Babu et al. (2021) applied the TOE as an overarching theoretical foundation when inquiring into data-driven innovative production in British manufacturing. All these applications serve to demonstrate that researchers consider the TOE to be a generic theoretical foundation for the DOI (Zhu et al. 2003).

One of the major variables that have emerged from the TOE model is technological compatibility and relative advantage, which are sub-dimensions of the technological context of MSMEs (Ramdani et al. 2013). The compatibility of an innovation with a business is defined as “the degree to which an innovation is perceived as consistent with the existing values, past experiences, and needs of potential adopters” (Rogers 2003, p. 240). Therefore, MSMEs’ adoption of a new technology introduces significant changes to the business infrastructure (Premkumar and Roberts 1999), which should be compatible with workplace practices, values, and the beliefs of individuals within organisations (Ramdani et al. 2013). At the same time, relative advantages affect technology adoption and diffusion in MSMEs positively (Cragg and Kind 1993; Ramdani et al. 2013). Relative advantage is defined as “the degree to which an innovation is perceived as being better than the idea it supersedes” (Rogers 2003, p. 229). Technological compatibility (Grover 1993) and relative advantage (Lee and Shim 2007) are the main components of the technological context of the organisation. They impact the adoption of new technology at MSMEs (Ramdani et al. 2013).

The TOE framework is abstract, and it has been linked to different constructs in empirical studies (Wang et al. 2010). The present study applies the TOE framework to the analysis of the various factors that influence the adoption of BDPA, which also proliferates the application of the framework. Thus, given the context of our work (Malaysian MSMEs), we rely on several studies that have isolated factors that can be classified into every dimension of the framework, be it the technological, the organisational, or the environmental. In our study, the technological dimension is represented by technological compatibility (Chen et al. 2015) and relative advantage (Wang et al. 2016b, a). The factors that comprise the organisational dimension are organisational readiness (Chen et al. 2015) and top-management support (Chang et al. 2016), while the competitive environment dimension is represented by competitive pressure and critical mass (Wang et al. 2016b, a).

Hypothesis development

Internal and external influences: the technology–organisation–environment framework and innovation diffusion

Compatibility

Technological compatibility plays a crucial role in the adoption of innovation. In a meta-analysis of 75 articles that goes as far back as the 80s, Tornatzky and Klein (1982) concluded that compatibility is the most cited factor in innovation adoption. IDT posits that the DOI is a process whereby an innovation is communicated between the members of a social system over a period of time (Rogers and Shoemaker 1971, 2003). Compatibility is the degree of congruence between the values of the members of the social system or innovation adopters and the innovation (Rogers and Shoemaker 1971). “Value compatibility” in Rogers (2003) is “normative” or “cognitive” compatibility in Tornatzky and Klein (1982). The latter proposed that, aside from normative compatibility, that is, compatibility with “what people think or feel about the technology”, there is also a more practical form of operational compatibility that is connected to “what people do”, or technological compatibility (Thornatzky and Klein 1982, p. 33). Premkumar et al. (1994) developed a similar dichotomy, distinguishing between two types of compatibility: the organisational and the technical. However, Tornatzky and Klein (1982) contended that the two types, cognitive and operational compatibility, are inextricably connected and related positively to the adoption of innovation. Similarly, Bunker et al. (2007) found that value compatibility is important for innovation adoption. It follows that the perceived compatibility of an innovation with organisational values and operational specificities affects decisions about adoption and its rate. Thus, we hypothesise as follows:


H1. Technological compatibility is positively related to big data and predictive analytics.

Relative advantage

Relative advantage is an important force in innovation diffusion. Rogers (1971, p. 213) defined it as “the degree to which an innovation is perceived as being better than the idea it precedes”. Potential adopters want to discover the benefits of adoption and compare them to the benefits of maintaining extant practices. In consequence, innovation campaigns are often framed to emphasise relative advantage Perceived relative advantage, that is, the perceived costs and benefits of adoption, is among the best indicators of the rate at which an innovation will be adopted (Rogers 2003). Moreover, the perceived benefits associated with adoption justify the decision to invest in an innovation. For example, Yapp et al. (2022) found that relative advantage encourages the adoption of ITs in small businesses. Mang’ana (2022) demonstrated its role in the adoption of Internet services on competitive advantage in the Kenyan banking sector. Wang et al. (2016b, a) found that the perceived relative advantage of mobile hotel reservation systems is a robust predictor of their adoption. Kapoor et al. (2015) revealed the importance of perceived RA in the utilisation of mobile technologies for interbank payments in India. More recently, Wong et al. (2020) established that relative advantage is an important determinant of the adoption of blockchain technology among Malaysian SMEs. The relative benefits also found in the ability of the firms to identify new business opportunities in product development in the sense of sustaining new product ideas (Cappa et al. 2021; Yasmin et al. 2020; Shan et al. 2019). Thus, we hypothesise as follows:


H2. Relative advantage is positively related to big data and predictive analytics.

Organisational readiness

Organisational readiness is the availability and utilisation of the resources needed for the application of a given technology (Iacovou et al. 1995; Chwelos et al. 2001). A trained and skilled workforce that is able to use BDPA is a key resource (Chen et al. 2015). Organisational readiness concerns technological and financial preparedness for adopting new technologies (Swatman and Swatman 1992). Organisational readiness has been found to be one of the positive determinants of the adoption of a technology at MSMEs (Ramdani et al. 2013). Thus, we propose that trained professionals who can work with big data and perform analytical tasks are key to organisational readiness. Accordingly, we formulate the following hypothesis:


H3. Organisational readiness is positively related to big data and predictive analytics adoption.

Top-management support

Previous studies suggest that organisational resources affect top-management support for innovative initiatives (Chen et al. 2015). Although the relative advantage of technologies can be appreciated by staff at all levels, it is the key decision makers who are vested with the legitimacy and political capital to make executive decisions about adoption (Wang and Dass 2017). Thus, the construct of top-management support is often used to investigate innovation diffusion. Top-management support has a positive effect on the adoption of innovation. Radio frequency identification supplies one salient example (Wang et al. 2010), as do cloud computing (Sayginer and Erkan 2020) and e-procurement in Jordan (Marei and Al-Jabaly 2021). Thus, we propose the following hypothesis:


H 4 . Top-management support is positively related to big data and predictive analytics.

Critical mass

An innovation has reached critical mass at “the point after which further diffusion becomes self-sustaining” (Rogers 2003, p. 343). Critical mass is a dynamic state in which so many actors in a system have adopted an innovation that a tipping point has been reached that innovation adoption has become self-sustaining. Critical mass is different from competitive pressure because the latter focuses on competition at the industry level. Critical mass is a phenomenon that occurs at the systemic level. Thus, in the context of MSMEs, systemic actors include not only competitors but also other important stakeholders, such as customers, suppliers, and partners. Critical mass research has found that changes in organisations are affected by the actions of other organisations that are similar in size. Critical mass is considered a key collaborative factor in innovation adoption (Byrd et al. 2021). Moreover, innovation between trading partners is encouraged, which is one of the benefits of critical mass (Ruutu 2017). Wang et al. (2016b, a) discovered that hotels are more likely to adopt mobile booking technologies if they perceive that their clients use mobile apps to arrange hotel reservations. In light of these findings, we hypothesise as follows:


H 5 . Critical mass is positively related to big data and predictive analytics adoption.

Competitive pressure

Competitive pressure concerns the rate of adoption of a given innovation at the industry level. The term refers to the degree to which an innovation is adopted in the firm’s industry. Research has shown that competitive pressure has a positive influence on innovation adoption (Premkumar and Roberts 1999; Yapp et al. 2022). The examples from empirical research include the adoption of e-business in European countries (Oliveira and Martins 2010), the application of enterprise resource planning systems (Pan and Jan 2008), and the utilisation of RFID utilisation in retail (Bhattacharya and Wamba 2018). In light of all this, we propose the following hypothesis:


H 6 . Competitive pressure is positively related to big data and predictive analytics adoption.

Competitive advantage

Competitive advantage is the ability to outperform competitors through the strategic utilisation of firm resources, skilled manoeuvring, and adaptation to the wider business environment (Porter 1980, 2008). A competitive advantage is attained when industry rivals are unable to replicate the strategies of a company. Competitive advantage is a function of the value that a firm is able to create for its clients and partners (Porter 2008). Value is what a buyer is ready to pay. It is higher when the item of consumption is of lower value but provides the same benefits to the consumer as those offered by competitors and when that item is unique and thus different from competing offerings (ibid). Competitive advantage is the ultimate goal of a firm (Porter 1980). Firms utilise different resources and capabilities to secure competitive advantages (Barney 1991).

In recent years, those resources and capabilities have increasingly come to originate from the realm of next-generation technology and analytics. The emphasis is on data “propelling insights toward competitive advantage” (Akter et al. 2020a, b). If a competitive advantage is to be sustained and if it is to generate value for the firm, its clients, and its partners, resources must be utilised and supported by the top management (Sirmon et al. 2007). BDPA have a wide array of applications, which were discussed in the literature review. Empirically, BDPA have been proven to improve decision-making (Hazen et al. 2014). Moreover, the adoption of BDPA has been found to have a positive effect on organisational performance and supply chain functions (Gunaserakan et al. 2017). The use of BDPA could help businesses to understand consumer behaviour (Ertemel 2015). Accordingly, we formulate the following hypothesis:


H7. Big data and predictive analytics are positively related to competitive advantage.

The moderating role of firm size

The participants in our research were employees of MSMEs in Malaysia. In our design, we rely on the OECD’s definitions. Accordingly, we assume that micro-firms employ between one and nine individuals, that small firms employ between 10 and 49, and that medium-sized firms employ between 49 and 249 (OECD 2019). The moderating role of firm size was acknowledged by the Austrian school of economics as early as the first half of the previous century (Schumpeter and Backhaus 2003). Larger firms were thought to have more political power and capital to invest in innovations and would thus champion innovative practices and industrial change. The same is not necessarily the case at present. High-tech industrial firms may well be more innovative than their older and larger counterparts (Shefer and Frenkel 2005). Some studies on e-tool adoption demonstrate that relatively smaller businesses are more agile (Levenburg 2005). There is an interesting parallel between size and organisational readiness as smaller firms are more ambidextrous. Because of their smaller teams, staff develop a wider portfolio of skills and expertise to deal with the day-to-day complexity (Boronat-Navarro et al. 2021). However, a recent study concludes that size matters when it comes to innovation adoption as smaller size means less capabilities and a greater need of Government support (Hong et al. 2016). Accordingly, we propose that firm size has a moderating effect on the relationship between the elements of the TOE and BDPA adoption. We test the hypotheses that are listed below to compare the rate of innovation diffusion in smaller and larger firms.


H 8a . Firm size moderates the relationship between technological compatibility and the adoption of big data and predictive analytics.


H 8b . Firm size moderates the relationship between relative advantage and the adoption of big data and predictive analytics.


H 8c . Firm size moderates the relationship between organisational readiness and the adoption of big data and predictive analytics.


H 8d . Firm size moderates the relationship between top-management support and the adoption of big data and predictive analytics.

H 8e . Firm size moderates the relationship between critical mass and the adoption of big data and predictive analytics.


H 8f . Firm size moderates the relationship between competitive pressure and the adoption of big data and predictive analytics.

The hypotheses and how they relate to each other are exemplified in Fig. 1.

Fig. 1
figure 1

Proposed model

Full size image

Research design and methodology

Methodology

The study is quantitative in nature. To test the hypotheses, a survey was chosen as the most effective means of reaching a wide range of participants. According to Ghouri et al. (2021), the quantitative approach is one of the most suitable approaches to research problems, that is, to testing hypotheses and theories and to examining the correlation between different variables and measures that are linked to the manner in which numerous individuals perceive a phenomenon. Hair et al. (2016) suggested that quantitative research enables investigators to collect data from a large sample to address specific questions. The latter is particularly true here because we attracted a reasonable number of responses from a range of companies.

Research design

Research design is an integral part of any research endeavour. It supplies the details for every step that is taken to achieve the goals of a study. The present paper examines the direct and indirect effects of technological compatibility, relative advantage, organisational readiness, top-management support, critical mass, and competitive pressure on adoption of big data and predictive analytics and competitive advantage. The research model and the hypotheses accord with the literature that was overviewed in the preceding sections. The sample was drawn from official databases, namely the SME corporation directory and the Federation of Malaysian Manufacturers. We collected data from the manufacturing and service sectors. We focussed on three types of firms: micro-firms (< 5 employees), small firms (manufacturing sector: between 75 and 200 employees; service sector: between 30 and 75 employees), and medium-sized firms (manufacturing sector: more than 200 employees; service sector: more than 75 employees; Al Mamun et al. 2021; Yusliza et al. 2020; SME Corporation Malaysia 2023). We investigated BDPA and CA because our main concern is with the competitiveness of Malaysian firms. The data were collected from the top and middle tiers of the management of firms in four provinces of Malaysia, namely Selangor, Perak, Perlis, and Kedah. The respondents were involved in using technology and data in decision-making to achieve strategic goals. We believe that managerial staff are the most suitable respondents because they possess relevant theoretical and practical knowledge (Ghouri et al. 2021). The questionnaire was distributed offline and online. A total of 202 responses were received from 166 firms. We excluded 56 invalid responses for several reasons, such as incompleteness, double check on scale, and the like. Table 1 presents the demographic characteristics of the sample.

Table 1 Demographics results
Full size table

Development of the instrument

The purpose of the study is to identify the antecedents and consequences of BDPA adoption in micro-, small, and large firms in the manufacturing and service sectors. For this purpose, valid, reliable, and standardised constructs and measurement scales were adopted from the literature. Before conducting the survey, we verified its content validity by consulting two experts, a seasoned management researcher and an ICT manager in the manufacturing industry in Malaysia. The researcher suggested amending two items for CA. Ultimately, both experts verified the contextualised psychometric properties. The suggestions were incorporated in the final version of the questionnaire. The questionnaire is divided into two sections: (a) demographic information and (b) items of the measurement constructs. The demographic part solicits information about the respondents and their firms, such as number of employees, industry, tenure, and the role of the respondent.

The second part consists of a technological dimension, an organisational dimension, an environmental dimension, CA, and BDPA. The technological dimension comprises two parts: TC and RA. Three items of TC were adopted from Chen et al. (2015), and three items of RA adopted from Wang et al. (2016b, a). The organisational dimension has two parts, TMS and OR. Three and four items of TMS and OR, respectively, were adopted from Chen et al. (2015). The environmental dimension covers two parts: CM and CP. Three and four items of CM and CP, respectively, were adopted from Wang et al. (2016b, a). Three items of BDPA were adopted from Chen et al. (2015), and six items of CA were adapted from Chang (2011). All items were measured on a five-point Likert scale, where 1 was the lowest value and 5 was the highest. Data privacy and confidentiality disclaimers were provided to potential respondents before data collection. The respondents were also informed that they were volunteers, that they could withdraw from participation at any stage, and that there were no presumed right or wrong answers to any of the questions. The details of the questionnaire are in Appendix 1.

Before the final analysis, we employed the independent t test method to check for nonresponse bias by comparing the first and last 25 respondents on all variables (Armstrong and Overton 1977; Chatterjee et al. 2021). Applications presented in Table 2 showed that there was no significant difference between early and late respondents, which indicates that there was no nonresponse bias (e.g. Kamble et al. 2021).

Table 2 BDPA applications
Full size table

Analysis

To analyse our data, we used version 3.3.3 of the SmartPLS software. SmartPLS is a statistical tool that enables the examination of a proposed measurement and structural model because survey data is not distributed normally, and it has the advantage of being able to accommodate small samples without requiring assumptions to be made about data normality (Chin et al. 2003). Multivariate normality was tested using the Webpower website, as suggested by Cain et al. (2017). The results show that multivariate skewness is 1.811 (p < 0.01), and multivariate kurtosis is 22.111 (p < 0.01), suggesting that the data are normal for multivariate skewness and abnormal for multivariate kurtosis. For this reason, we ran the bootstrapping procedure to generate standard errors when testing the structural model.

We tested for common method bias by examining full collinearity (i.e. Akhtar et al. 2022; Sarstedt et al. 2020). In this method, all variables are regressed on a common variable. If the variance inflation factor (VIF) is less than or equal to 5, then there is no bias from the single data source. As shown in Table 3, the analysis yielded a VIF of less than 5. Therefore, common method bias is not a serious risk in this study.

Table 3 Collinearity analysis
Full size table

Measurement model

The measurement model was assessed by examining loadings, average variance extracted (AVE), and composite reliability (CR; Ghouri et al. 2021; Sarstedt et al. 2019). Specifically, three criteria were assessed. First, all indicator loadings should exceed 0.6. One item, CP2, was dropped due to low loading (0.482). For each construct, AVE should be greater than 0.5, and CR should exceed 0.7. As shown in Table 4, all indicator loadings are above 0.5, CR values range between 0.792 and 0.833, and AVE ranges from 0.547 to 0.739. Cronbach Alpha values range between the 0.779–0.828. All three conditions for reliability and convergent validity are thus met. We tested for heterotraitmonotrait ratio (HTMT) discriminant validity (Ghouri et al. 2020; Hair et al. 2017) with a threshold value of 0.85, and the constructs are distinct—as shown in Table 5, all ratios are lower than 0.85. Therefore, discriminant validity was also confirmed for each construct.

Table 4 Measurement model
Full size table
Table 5 Discriminant validity (HTMT)
Full size table

Structural model

We analysed the path coefficients, standard errors, t values, and p values for the structural model using a 5000-sample-resample bootstrapping procedure (Henseler et al. 2015; Hair et al. 2019). In addition, alive to the criticism of Hahn and Ang (2017), who argued that p values are not a good criterion for testing the significance of hypotheses, we employed a combination of criteria, such as p values, confidence intervals, and effect sizes. Table 6 shows the summary of the criteria used for hypothesis testing. As shown in Table 6, Hypotheses 1–7 are all supported by our findings. The in-sample explanatory power total R2 of the results is 38.1%.

Table 6 Hypothesis testing (H1–H7)
Full size table

The moderation analysis that is presented in Table 7 reveals that H8a, H8b, H8c and H8d are partially supported, H8e is not supported, and H8f is supported.

Table 7 Moderation results and hypotheses testing (H8a–H8f)
Full size table

H8a is tested the relationship between technological compatibility and big data and predictive analytics varies across firm sizes, in that micro-firms and small firms differ significantly from medium-sized firms. The difference between the beta values for micro-firms and small firms is 0.001 (p > 0.05). The difference between the beta values for micro-firms and medium-sized firms is 0.186 (p < 0.01), and the difference between the beta values for small firms and medium-sized firms is 0.294 (p < 0.01). H8b is tested the relationship between relative advantage and big data and predictive analytics varies across micro-firms and medium-sized firms, in that micro-firms differ significantly from medium-sized firms. The difference between the beta values for micro-firms and medium-sized firms is 0.096 (p < 0.05). The difference between the beta values for micro-firms and small firms is 0.000 (p > 0.05), and the difference between the beta values for small firms and medium-sized firms is 0.001 (p > 0.05). H8c is tested the relationship between top-management support and big data and predictive analytics varies across micro-firms and small firms, in that micro-firms differ significantly from small firms. The difference between the beta values for micro-firms and small firms is 0.117 (p < 0.05). The difference between the beta values for micro-firms and medium-sized firms is 0.000 (p > 0.05), and the difference between the beta values for small firms and medium-sized firms is − 0.006 (p > 0.05).

H8d is tested the relationship between organisational readiness and big data and predictive analytics varies across micro-firms and medium-sized firms, in that micro-firms differ significantly from medium-sized firms. The difference between the beta values for micro-firms and medium-sized firms is − 0.116 (p < 0.01). The difference between the beta values for micro-firms and small firms is 0.012 (p > 0.05), and the difference between the beta values for small firms and medium-sized firms is 0.004 (p > 0.05). H8e is tested the relationship between critical mass and big data and predictive analytics is not varying between types of firm sizes. The difference between the beta values for micro-firms and medium-sized firms is 0.000 (p > 0.05). The difference between the beta values for micro-firms and small firms is 0.001 (p > 0.05), and the difference between the beta values for small firms and medium-sized firms is 0.000 (p > 0.05). In last, H8f is tested the relationship between competitive pressure and big data and predictive analytics varies across firm sizes, in that micro-firms and small firms, and micro-firms and small firms differ significantly from medium-sized firms. The difference between the beta values for micro-firms and small firms is 0.163 (p < 0.01). The difference between the beta values for micro-firms and medium-sized firms is 0.154 (p < 0.01), and the difference between the beta values for small firms and medium-sized firms is 0.191 (p < 0.01).

Discussion

Summary of results

The study examined the prevalence and primary applications of big data and predictive analytics (BDPA) practices among micro-, small, and medium enterprises (MSMEs) in Malaysia. The research also aimed to compare the use of BDPA among enterprises of different sizes and to investigate the relationship between BDPA and competitive advantage. The findings revealed that microbusinesses are the most significant beneficiaries of BDPA, followed by small and medium-sized businesses. The most common applications of BDPA were for customer relationship management (CRM) and warehouse and operations management, highlighting the potential benefits of using BDPA in streamlining business operations and improving customer satisfaction.

Interestingly, the study found a strong correlation between the use of BDPA and competitive advantage. Businesses that used BDPA were more likely to have a competitive edge in their respective markets. However, despite the advantages of BDPA, the study found that less than 20% of medium-sized businesses in Malaysia utilise this technology. In contrast, around 33% of small businesses use BDPA, indicating a significant gap in the adoption of BDPA across different enterprise sizes.

Overall, the research provides valuable insights into the prevalence and applications of BDPA practices among MSMEs in Malaysia. The findings highlight the potential benefits of using BDPA to improve business operations and gain a competitive advantage in the market. The study also underscores the need for greater awareness and adoption of BDPA among medium-sized businesses, given the potential benefits that this technology can provide.

Theoretical contributions

In this paper, we studied the direct and indirect effects of technological compatibility, relative advantage, organisational readiness, top-management support, critical mass, and competitive pressure on the adoption of big data and predictive analytics, and on competitive advantage in the context of Malaysian businesses. We examined the data collected from micro-firms, small firms, and medium-sized firms. Our focus on MSMEs was driven by the existing gap in the literature where not enough attention was paid to smaller emerging marker firms that are using BDPA.

Two of our findings are noteworthy and contribute to the field of innovation adoption among MSMEs. First, we confirm the role of TOE factors in IDT, specifically in the context of Malaysian MSMEs. Innovation diffusion is the process by which a technology spreads across a population of organisations (Fichman 2000; Rogers 2003). Our study demonstrates that indeed, technology is diffused when it helps address the needs of individuals and groups and also when other key players in an industry adopt innovations to achieve competitive edge. Regardless of firm size and industry, organisations are evolving and adopting the new technologies to stay competitive. This is an important aspect of the nature of competition, as firm size is often regarded as a determining factor of competitiveness. As innovations, BDPA are perceived as an antecedent of competitive advantage by MSMEs of all calibres. This finding is consistent with previous BDPA studies. In the changing global technological environment, a firm’s ability to integrate, build and reconfigure its competences are key (Dubey, et al. 2019; Teece et al. 1997). Regardless of the universal benefit of BDPA, we found that size does matter for the intensity of use and application of BDPA. Malaysian microenterprises use BDPA more than small and medium-sized enterprises. It appears that micro-firms in Malaysia are more flexible and adaptable. This is consistent with Levy and Powell’s (1998) argument that SMEs are generally more flexible and adaptive than larger organisations. Our findings are also consistent with Boronat-Navarro et al. (2021) assumption that smaller companies have greater agility and organisational ambidexterity.

Secondly, our study also offers several insights on the relationship between size and innovation adoption. Our results reveal that medium-sized firms use BDPA the most, followed by small and medium-sized enterprises, and micro-firms. It appears that technological compatibility grows in importance with firm size. This result is aligned with the findings of the study of Hong et al. (2016). Firm size usually related to the ‘production capacity’ is seen as highly correlated with innovation (Armbruster et al. 2008). One explanation is that larger companies invest in infrastructure and personnel, and novelties must be compatible with existing practices and expertise. Similarly, medium-sized firms investing in mobile integrated system to gain relative advantage. This advantage provides edge in new offering to an existing one. Furthermore, in small enterprises, the support of managers and the available resources and skills play a more decisive role in innovation diffusion than they do at micro- and medium-sized firms.

Top-management support plays a more significant role in small enterprises than in micro- and medium-sized enterprises. The possible reason of this finding may lie in the specific characteristic—smaller number of employees and structure of small enterprises (e.g. Blili and Raymond 1993). Smaller teams of people are closely managed by more influential managers. Organisational readiness is more important to micro-firms than to small ones. As a pre-requisite of innovation adoption, organisational readiness is an important state that organisations need to maintain and strive towards, when pursuing competitive advantage. This could be explained with Levenburg (2005) suggested that principle of greater agility of relatively smaller firms when it comes to adopting e-tools. Our findings clearly demonstrate that organisational readiness (pre-existing skills and resources) is not as important as a facilitating factor, as compared to their larger counterparts. Competitive pressure exerted an important influence on all of the MSMEs that we engaged. However, medium-sized firms are more influenced by their competitors than micro- and medium-sized ones. Interestingly, the competitive environment influence that reflects wider BDPA adoption across an industry and the economy, did not vary between different calibres of MSME. Sun et al. (2020) found that competitive pressure forced firms to adopt technologies like big data, regardless of firm size. This is also consistent with an earlier study of Matthyssens and Van den Bulte (1994) who conclude that increased competitive pressure forced them to innovate more quickly and to provide a higher quality and reliability.

Managerial implications

Our analysis has several implications for professional practice at MSMEs. First, our study provides a preliminary overview of the adoption of BDPA in Malaysia. This overview could be useful in competitor analysis or in wider business environmental mapping. We believe that this knowledge would be useful both to Malaysian MSMEs and to MSMEs from other newly industrialised economies in the region, more specifically other ASEAN countries. Second, our study isolates some of the internal and external factors that influence the adoption of BDPA. MSMEs could benefit from our research by reflecting on their internal environments, whose elements include top-management support, organisational readiness, and technological compatibility. Similarly, our study would benefit those who must analyse wider competitive environment factors, such as critical mass and competitive pressure. Thus, we believe that the results that we have presented can aid MSMEs in their decision to adopt BDPA and achieve improvements in performance.

BDPA requires expertise that is not always available at organisations; more often than not, it is outsourced. Thus, our research has wider implications for IT consultants in Malaysia, who may be willing to build the skillset that is necessary to support MSMEs. Our results clearly demonstrate that MSMEs find BDPA advantageous. However, skills, organisational resources, and top-management support are all factors that could either facilitate or obstruct the use of BDPA. Finally, our findings could be useful for high-level decision makers, who could develop national and industrial policies that facilitate and incentivise the adoption of BDPA. The most commonly cited beneficial uses of BDPA, such as product and process optimisation, purchase analytics, and supplier sourcing, did not appear to be used frequently by our respondents. BDPA were mostly used for customer relationship management, operations improvement, and warehouse management. It appears that BDPA can develop and proliferate in different operational fields. Although we explored some common uses of BDPA, our list is by no means exhaustive. For instance, waste reduction, a pressing issue in rapidly industrialising countries, could be addressed through the use of BDPA (Tayal and Singh 2018). Similarly, sustainable supply chain management could be improved with the assistance of committed governments (Rentizelas et al. 2020).

Conclusion

This study allowed for the drawing of several lessons, presented in this section. First, the adoption of big data and predictive analytics in Malaysia is still at its infant stage. Second, while still in its initial development stages, where utilised, BDPA have affected business operations and profit. Third, a refined understanding of the factors that facilitate or obstruct the adoption of big data and predictive analytics could further our understanding of innovation diffusion and lead to other benefits.

The findings presented above are subject to a caveat as all research endeavours have their limitations. Our data were collected in Malaysia, a rapidly growing, newly industrialised economy. This section describes some of the other limitations of our research. The results and the findings reflect our effort to understand the antecedents and consequences of big data and predictive analytics. This study shows the firm sizes differences in approaches while adopting different initiatives to sustain and compete in market. Despite the sizable practical implications, this study has several limitations that open avenues for future research. First, the study drew on data collected from the 12 sub-sectors of manufacturing and services. Future research could concentrate on other sub-sectors or another sector to derive more profound insights.

Our findings present initial evidence on the sectors that do use big data and predictive analytics as well as on the main purposes that big data and predictive analytics serve. We believe that this exploration would indicate to others that there is a vast avenue for future BDPA research in emerging economies, like Malaysia. To arrive at our findings, we relied on a theoretical framework that integrates innovation diffusion theory and technology–organisation–environment. We believe that there is scope for more research projects that are informed by the use of these frameworks. Thus, this study is grounded in established research while catalysing future inquiries into use of analytics at MSMEs.

The literature avers to other antecedents and consequences of big data and predictive analytics, such as privacy, trust, customer engagement, supplier engagement, purchase behaviour, return on investment, and others. Therefore, future studies can apply and improve the theoretical model that we used to examine the antecedents and consequences of big data and predictive analytics. Last, we recommend applying the extended model to a multi-group approach to government departments. This extension would enable an analysis of the attitudes of key decision makers towards big data and predictive analytics and a study of its impact on competitiveness and operations management. In last, further investigation is needed to determine whether competitive pressure ought to be attributed to low critical mass in Malaysia.