1 Introduction

Big data is poised to change the way we live and work (Manyika et al. 2011; Mayer-Schönberger and Cukier 2013). The implications of deploying big data for work, such as in smart city initiatives, has impacted the way organizations operate. Organisations have now had to redefine and construct new models to adapt to this disruptive technology. George et al. (2014) stressed that even though the term big data has become a common business parlance, there has been little research in management scholarly circles. Especially those that address “the challenges of using such tools—or, better yet, that explores the promise and opportunities for new theories and practices that big data might bring about” (p. 321). There has also been little organizational research that has explored the learning experiences in the advent of such new technologies. Therefore, this research explores the learning processes involved in how public organizations have been able to embrace big data in providing smart city solutions.

The smart city concept is widely perceived as a means of solving urban issues through the integration of information technology (IT) with the city’s infrastructure (Caragliu et al. 2011). Big data has, in turn, become an integral part of this phenomenon. As this study is time-sensitive, there are difficulties defining the constituent parts of the research context: namely, big data and a smart city. As such, working definitions of concepts are constructed for this paper. Therefore, big data is the generation of infinite, unstructured data from different sources that possess diverse characteristics. On the other, urban issues are regarded as wicked problems—intractable problems (Bettencourt 2014; Rittel and Webber 1973). Whereas, a smart city is one where “investments in human and social capital and traditional (transport) and modern (ICT) communication infrastructure fuel sustainable economic growth and a high quality of life, with a wise management of natural resources, through participatory governance” (Caragliu et al. 2011, p. 70). These definitions are appropriate because they capture the current state-of-the-science and help define the boundaries of this research. Invariably, the smartness of a city has been linked to how it deploys and integrates big data as a tool used in the delivery and provision of services (Batty 2013; Kitchin 2014). The data used in smart city initiatives are usually public data that can be accessed, in some instances, under restrictions to protect privacy, for example in areas energy use, healthcare and transportation (George et al. 2014).

Organisations have found themselves at the centre of a data deluge. So far, Tambe (2014) has found out that firms’ that invest in big data technologies have a higher productivity are than those that invest less. Whereas, Popovič et al. (2016) point that the use of big data enhances decision making and business performance in manufacturing. To the best of our knowledge, we have found no research that has explored how manager’s public or private organizations learned to deploy the use of big data to carry out organisational tasks. We look to research in organizational learning (OL) to consider how organizations exploit and experience using big data, in this case for carrying out smart city initiatives. For consistency with the literature and clarity in presentation, we adapt the definition of organisational learning to be the process of building capacity for effective organizational action through knowledge and understanding (Burnes et al. 2003; Carroll and Edmondson 2002; Elkjaer 2004). We argue that given the fast moving pace and the novelty of such concepts, organizational efforts to understand what they are, what they can do, and the infrastructure needed to exploit and manage it from a public sector’s perspective, so far, are based on an organizations’ ability to embed acquired knowledge.

In a technology and data-driven world, learning as an organization becomes particularly critical to its survival, performance and success. In addition to arriving at a working definition of organizational learning, we seize this opportunity to use Crossan et al.’s (1999) 4I multilevel learning framework to study the learning aspects of the use of big data in public sector organizations for creating smart cities. The 4I framework is a dynamic tool that is adaptable to different management research domain: from leadership (Vera and Crossan 2004) to strategic renewal (Crossan and Berdrow 2003; Jones and Macpherson 2006). The 4I is central to our work because it provides a unique opportunity to explore how organizations learn about the use and the introduction of big data in smart cities. As such, we ask how public organizations have been able to embrace big data for addressing urban issues and what lessons have been learned from the activities that have been embarked upon.

The central findings centre on the fact that the advent of big data was accompanied by difficulties—i.e. knowledge and power dynamics—which challenged the way things had previously been done in the respective city authorities. The findings also highlight the resources at the disposal of public organizations to carry out smart city schemes provides the leadership, and, most importantly, the opportunity to learn from initial engagements with big data in order to carry out broader initiatives. We make four key contributions to the big data and organizational learning literature. First, (knowledge finding) we argue that the provision of organizational resources is pivotal to organizational learning with regards to reframing problems that an organizations looks to address. The adoption of new technology, such as big data, is underpinned by the availability of the necessary funds. In other words, the necessary finance to experiment with smart city initiative determines how audacious, in terms of scope. Second, (knowledge framing) we argue that data-driven policies, in these cases urban policies, require a fundamental shift and alignment with broader organizational policies. Third, (inter-organizational collaborations) in the era of big data, the understanding of problems facing an organization will need to be carried out by reframing knowledge and proactively collaborating with other organizations to acquire and embed knowledge. Last, (ex-post evaluations) reflections on past experiences impact the future practices, which is a form of reflexive practice. These findings provide the groundwork for organizations for learning how to use and introduce big data.

This paper is structured as follows. First, we discuss the academic debate on the use organizational use of big data. Second, we provide an overview on organizational learning through the lens of the 4I framework. Third, the research’s setting and methodology are also outlined and analysed. Fourth, using two smart city initiative cases in the United Kingdom (UK), the data collected from three sources—semi-structured interviews, observations and secondary data—are analysed. Fifth, we present findings and discuss the implications of the findings, encapsulated in an experiential model. Last, we conclude with the limitations and potential areas for future research.

2 Literature review

2.1 Big data

Big data, as the name implies, are data in complex terms of size, but also in variety and relativity to other sources of data; making it difficult to analyse with conventional database management techniques (Manyika et al. 2011). It is data generated from a growing variety of sources ranging from clicks on the internet, mobile transactions, business transactions, user-generated content, social media, as well as purposefully generated content through sensors or genomics, healthcare, engineering operations management, finance and industrial internet (George et al. 2014). It is increasingly becoming an important tool used by organizations to create value. As such, big data is a key element in developing smart city solutions (Batty 2013; Kitchin 2014).

According to the International Data Corporation (IDC) (2011), big data is a new generation of technological architecture that can extract economic value from very variety of huge volume of data; thereby, capturing of a high velocity, discovery and analysis of data. Big data analytics is the way an organisation deploys its computing infrastructure to analyse and validate high volumes and velocity of data to create value (Agrawal et al. 2011). Big data also has to deal with the way organisations manage unforeseen content with an unknown structure and the enabling of real-time collection of analysis (Villars et al. 2011). Incidentally, big data is gaining a strategic importance to organizations (Constantiou and Kallinikos 2015; Yoo 2015). Tambe (2014) explored how the technical human capital—the technical-know-how of a firm’s labour force, in this case in Information Technology (IT)—determines the adoption point and levels of investment in big data technologies. Tambe draws a parallel on the adoption of big data with research and development (R&D) external investment in firms through the way investment levels impact a firm’s R&D efforts. Investment in IT brings about significant benefits to R&D. The conclusion arrived at was that labour market concentration was not significant but instead, the initial levels of investment in big data technologies assured the firm of higher productivity. This underlines the importance of investments and technical abilities in the deployment of big data.

As a result, organizations do not have to replace their strategic tool boxes. Rather organizations can use the existing tools at their disposal more effectively (Woerner and Wixom 2015), to capture more value from big data and to inform existing tools. Bhimani (2015) argue that big data has altered the information flow and volume within the organization. To this end, big data shapes an organizations strategy and shapes how it is used for value creation. For example, smart cities; the ability of cities to collect and analyse large amounts of structured and unstructured data enables them to make data-driven decisions for the provision of amenities and services through smart city initiatives (Marsal-Llacuna and López-Ibáñez 2014).

Big data equips stakeholders with previously never untapped insight into organizational problems. Within the smart city domain, depending on the initiative a city has embarked on, data can be gathered from different sources. There are six main characteristics of a smart city: (1) smart economy; (2) smart people; (3) smart governance; (4) smart mobility; (5) smart environment and (6) smart living (Giffinger et al. 2010). Big data can be used to address problems that stem from these characteristic areas. Chen et al. (2012) drew up a general schema of areas where big data can be applied in society; from fostering citizen participation with e-government initiatives, also referred to in the smart city literature of smart governance, to smart health and well-being, also referred to as smart living.

In general, the concept of big data has grown as a result of a wider variety of sources and characteristics of data. The data gotten from these various sources serve as a tool for problem-solving across different aspects of business and society. The utilization of big data is determined by how organisations access and analyse structured and unstructured as well as mixed data from various sources, as well as, having the appropriate infrastructure to gain a competitive edge. Thus, the ability of organizations to harness their big data potential will determine their survival (long and short), performance and success. Given the context of the research, city authorities are privy to a huge number of granular data generated from its citizens. Given the access some cities have to such data, deploying such a tool has become central to making work smarter and efficient. Organizational learning can be central to this.

2.2 Organisational learning: a 4I model perspective

Organizational learning (OL) is a complex process, which is important to the long-term survival of a firm. In general, OL is defined as the process by which organizations learn especially in a fast changing environment (Chiva et al. 2007). Even though studies in OL have flourished, its broad definition brings with it criticisms. There has been too much of a consensus on OL means with little nuance (Wang and Ahmed 2003). Regardless, OL is still the collective process of acquiring and creating competencies that are modified by the way in which situations get managed and transformed as a result of this intrinsically complex concept, it is imperative to integrate or adopt a multilevel approach (Argote and Miron-Spektor 2011; Steven and Dimitradis 2004). Indeed, the strength of the OL concept potentially lies in the wide array of areas in which it can be applied.

Initial studies on OL focused on utilizing experiences to enabling competences (Argote and Epple 1990; Yelle 1979). Recent studies have examined the effects on organizational performance (Argote and Miron-Spektor 2011). Although a large body of literature has been established in the OL literature, relatively new studies explore how organizations learn in the advent and deploying of new technology. Conceptually, Andreu and Ciborra (1996) assess learning using IT with a resource based view framework (RBVF). Andreu and Ciborra (1996) assess the aspects of learning with regards to capability development by exploring how information technology (IT) contributes to it.

Furthermore, OL has been investigated from different perspectives. OL can be studied irrespective of the size of the firm (Matlay 2000). Learning could occur within and between organizations, irrespective of if they public or private entities (Beeby and Booth 2000; Maden 2012). Most importantly, in order for this process to manifest, the firm should have the capability to learn. Steensma (1996) particularly note that inter-organizational collaborations are a pivotal (mediating) role on firm’s ability to acquire technological competencies. Wang and Ahmed (2003) suggest that researchers are able to explore the concept on different levels within an organization. In turn, it enables researchers to assess the systems and processes that facilitate learning in an organization. It can also be embedded with other management concepts and phenomena, such as culture, creativity, entrepreneurship, knowledge management and innovation (Matlay and Mitra 2002; Wang and Ahmed 2003). However, a more encompassing framework is the 4I model (Crossan et al. 1999).

2.3 The 4I model

The 4I model by Crossan et al. (1999) was built based on previous work by Argyris and Schon (1996), Nonaka and Takeuchi (1995) and Kim (1993). The collective knowledge is built four sub-processes—intuition, interpretation, integration and institutionalization—which take place across individual, groups and organizational levels (see Fig. 1). These dimensions of this OL process are illustrated in Fig. 1: the 4I model. The model is based on the premise that an organization must manage tensions between what had been previously institutionalised as against the emergence of new knowledge.

Fig. 1
figure 1

The social Psychological and political processes of organizational learning

Full size image

Irrespective of the level and unit of analysis, the 4I models takes first intuition, through the exploration or assimilation of new knowledge. Intuition is presented as the feed forward in the cells of the upper part of the diagonal matrix in Fig. 1. Second, it is a reductive logic that exploits what has been learned from the feedback cells (see lower part of the matrix in Fig. 1). Thus, feeding forward learning serves as the knowledge that emanates from individuals, which is then transferred to groups and spread across the organization. On the other hand, learning occurs from an organizational level to groups and individuals through a feedback process. Integration ties the two types of learning, which pivotal to the institutionalization of the exploratory results and the eventual interpretation by groups and individuals of institutionalized learning (Crossan et al. 1999).

The 4I model is consistent with previous works that explain the links between individual, group and organizational levels of interaction (see Nonaka and Takeuchi 1995). It presents OL as a process where the stages are identified; the interactions among the different levels in the organization recognized; the influence wielded by respective individuals in the routines observed; the effects of feedback and the interpretation that accompany the processes. The 4I model has been used to explore various forms of organizational learning. For example, Real et al. (2014) in using the 4I model, found out that organizational learning plays a mediating role in a firm’s entrepreneurial orientation. Also, Dutta and Crosson (2005) used it develop an understanding of the nature of entrepreneurial opportunities in organizations.

The 4I model has also been used in examining the relationship between absorptive capacity and organizational learning (Sun and Anderson 2010). This was based on the notion that existing organizational processes for learning are based on a combination of the stock and flow of knowledge. Furthermore, Schulze et al. (2013) operationalized the model in mapping the value stream for product development, while Stevens and Dimitriadis (2004) carried out a longitudinal study on learning from new service development. Similarly, Berson et al. (2006) and Vera and Crossan (2004) deployed the model in order to assess leadership in relation to organizational learning, while Jones and Macpherson (2006) extended the model to explore inter-organizational learning in relation to strategic renewal.

The OL 4I model is critical to this work because, first, it helps in the exploration of an interpretation that cuts across leadership, culture, resources and organizational structure. Second, it demonstrates that innovation and learning go hand-in-hand and cuts across the different levels of the organization. Finally, the complexity and novelty of the concepts of big data and smart cities requires a model that can capture the dynamics of the learning process with big data.

3 Methodology

3.1 Research setting

Two case studies were used for this research. Both smart city initiatives have since been launched as demonstrators that, in every sense of the word. Demonstrators illustrate what can be achieved with the deployment of big and open data as tools for achieving smart city status. Even though the impacts of these demonstrators are still being assessed (ex post), these experiences give valuable insight into the workings of big data for addressing urban issues. We ensured that these projects had gone live and had lived through, at least, half of its lifecycle.

3.2 Case 1

The first case is a smart city initiative based in the south-west of England. It aims to use smart technologies and digital connectivity to meet the city’s environmental, social and economic challenges and opportunities and become a truly smart city (Doc.A7). The open data platform is a key part of this smart city initiative that seeks to open up access to the city’s data in order to make it easier for citizens, researchers and developers to access, analyse and share information (Doc.A7). The categories of data are to be made accessible for citizens, researchers, organisations and developers (Doc.A8). The city was able to secure a £3million grant and is actively working on securing other structured financing.

The smart city under study seeks to open access to its data to make it easier for citizens, researchers and developers to access, analyse and share information. Making these data streams available is to enable the development of new solutions for tackling urban issues (Doc.A28). In this case study, open data has given the citizens a unique opportunity to participate in the decision-making of the city. Furthermore, smart city intends to use smart technologies and digital connectivity to meet the city’s environmental, social and economic challenges and opportunities and become a truly smart city.

Making these data streams available will enable the development of new solutions for tackling urban issues. The rationale behind it is that it will allow new solutions for the city’s problems to be developed. The process is carried out through an inter-institutional arrangement with public and private organisations as well as the general public. The initiative aims to ensure that the city benefits from a world-class, inclusive, green and digital economy. How does the city intend to achieve these goals? This would be by: (a) increasing the city’s resilience and self-sufficiency through local energy generation; (b) working with others to deliver targeted support to key business sectors; (c) attracting investment in digital infrastructure (Doc. A16).

3.3 Case 2

The second case is a smart city initiative based in a city in Scotland. The aim of the initiative is to demonstrate how technology can make living in the city smarter, safer and more sustainable (Doc.B19). In doing so, the city authorities and planners are putting residents at the forefront of the technological integration and application. It is a data-driven process that is meant to assist policymakers and inform future investments. The initiative addresses the challenges facing the city such as health, safety, transport and sustainability (DocB.16). Adopting an inter-institutional arrangement that involves the public, academics and businesses, which is geared towards getting these stakeholders are involved in using the data and contributing their own knowledge to the initiative. The funding for this initiative was a grant obtained by the city to be spent over a period of three years (DocB.4).

This case study focuses on the two already running elements of the initiative: the open data and demonstration projects work streams. The open data work stream deploys an intelligent data platform to store, analyse and publish real-time data on an online dashboard (DocB.19). These can be accessed on widgets and smartphone applications (apps) (DocB.11). For example, one of the apps allows users to bring to the attention of city authorities uncollected trash and potholes—by tagging the potholes on their smart phones—as well as following the progress of the issue that was reported. Also, in line with open data principle, by opening up data to the public, the city is able to engage with entrepreneurs and application developers who have come up with useful ideas and solutions that help the city address urban issues. To date, more than 400 data streams have been identified in the city; they include information on everything from bin collections to footfall in retail areas.

The demonstration project has four key objectives that address some of the challenges faced in the city: active travel, energy efficiency, integrated social transport and intelligent street lighting. For active travel, there is a cycling app that records the journey of cyclists, so that the council knows what routes are regularly used, which in turn, allows them know where and how to channel resources towards having an adequate cycling infrastructure. From inception in 2013, the city has embarked on building three-dimensional (3D) model sensors in public housing buildings to help improve the energy efficiency of the citizens and city (DocB.22). By using digital monitoring services to optimize the use of the authority’s vehicles—that transport the elderly and children to and from appointments—were integrated to enhance social transportation. The council have demonstrated that they are able to redeploy these vehicles to other divisions when idle. Last, through the deployment of intelligent street lighting, the city has been able to measure the air’s pollution, lighting as well as footfall (DocB.22).

3.4 Research design and data collection

Given that our goal was to understand the learning experiences and processes involved in using big data for creating smart cities, our review revealed that there are unanswered questions within the OL literature and 4I model; specifically on how organizations learn to adopt new tools, such as big data, for new concepts, such as smart cities. Due to the domain gap, we adopt an inductive grounded theory (Corbin and Strauss 2008; Edmondson and McManus 2007). Furthermore, an inductive approach is more suitable for addressing “how” questions (Creswell 1998).

As outlined in Table 1, the study relied on both primary and secondary data sources: (a) semi-structured interviews with elite top organizational actors involved in the planning and execution of the data-driven smart city initiatives; (b) secondary data, documents, found in various media, such as newspaper articles, smart city initiatives websites, publications on the case by city authorities, as well as documentation specifically requested from the participants; (c) we observed the demonstration—showing how it works—of some of the schemes within the initiatives. Two field diaries were also maintained: one to record informal discussions and the other to record participant observations.

Table 1 Overview of data collected
Full size table

The primary data was collected from elite participants—top management—in the respective city councils from July 2015 through to October 2016. There were a total of 10 elite participants in this study. In general, given that the use of big data in public organizations is still at its infancy, the amount of participants is significantly modest. Overall, we conducted semi-structured interviews for an hour each with 1 politician, 2 city executives and 4 project leaders (see Table 1). In order to ascertain the encompassing impact of the use of big data for smart city solutions, the questions that were asked revolved around ‘how things were done before big data?’; ‘how things were currently done?’; and ‘how things will be done going forward?’. The sources of secondary data were pivotal in triangulating the self-reportage from these key elite-participants, which, mitigates against potential bias—retrospective or otherwise—in the interviews conducted with these organizational actors.

Given the aim of the research, we identified cities that were at the forefront of the smart city idea. We began by contacting the city councils, where we were then directed to their key project contacts. After informal interviews, formal interviews were held with those contacts. Adopting a snowballing process (Lincoln and Guba 1985), we were referred to other elite participants that were involved in the smart city initiatives. This was important for making that transition from purposeful sampling (Patton 1990) to theoretical sampling from the data collected with the use of the theoretical model. For instance, when concepts that had the potential to give theoretical insight emerged (e.g. those surrounding learning about the machinations of big data), we then focused more in-depth in order to unearth details on the processes and experiences around the phenomenon under study. This cycle continued to the point of theoretical saturation: that point where data gotten from the participants or secondary data offered no new insight or conceptual categories. Participants in the first case offered to illustrate some of the demonstrators in addition to the interviews. On the other hand, the stakeholders in the second case referred us to useful websites.

3.5 Data analysis

The interpretative approach formed the basis of the analysis. This is because we were interested in exploring and understanding from the perspective of key organizational actors the unfolding of events (Morgan and Smircich 1980). With this approach, the process requires a fair amount of comparability that can be richly described (Langley 1999). This approach is, therefore, in line with the interests in understanding the learning experiences and processes involved with using big data for smart cities. The experiences to date between both case studies provided a comparability function. After using secondary data to construct a broad overview of the cases, the learning points were derived by, initially, reviewing the interview transcripts. This led to the identification of units within the interviews that represented important ideas and concepts. We were cautious not to be swayed and led by these high-profile participants. As is the problem with interviewing elite participants, as noted by Smith (2006), most of them are accustomed to public speaking and engagements—one of the participants has had a TEDex Talk. They could also see it as incumbent on themselves to promote the organizational interests through this mode. For consistency, with the OL literature, we began with identifying theoretically relevant themes. A narrative approach was used analyse the interviews. All the interviews were tape recorded, transcribed and transferred to the NVivo software for data management and coding.

According to Reissman (1994), a narrative analysis is a “systematic study of personal experiences and meaning: how active subjects have constructed the events” (p. 78). It is, therefore, an interpretative technique that pays attention or focuses on the stories or narratives people render about their experiences. The aim is not to find out if the accounts are factual or accurate, but rather attention was paid to how and why participants construct the kinds of narratives they produce. In order to get a holistic and content specific perspective, a composite narrative was created. Given there has been a considerable work around organizational learning, especially with a multi-level model, such as the 4I, narratives was useful in generating new insights.

In constructing these composite narratives, the discourse with the elite participants sought to understand their experiences on working with big data for smart cities (Dunford and Jones 2000). According to Currie and Brown (2003), composite are a result of discourse between a researcher and the individuals discourse. It should be noted that narratives tend to be elaborated haphazardly (Boje 2001). As a result, the questions asked to the participants were framed to capture the way things were done before the advent of big data, then proceeded to capture how things were presently done, and finally, how they envisaged things will be done. Field diaries were used to refine and reflect upon the emerging themes (Miles and Huberman 1994).

3.6 Cross-case analysis

The use of two case studies was useful in examining aspects of the discourse around the use of big data. In some instances, participants drew comparisons between themselves and those in the opposing case study (they were not made aware of themselves). For consistency and clarity, this qualitative study was iterative in nature (Miles and Huberman 1994). Thus, the narrative analysis allowed for the assessment of the elite participants perspectives on the use of big data. From these, we were able to pull out the learning points so far on the use of big data.

To this end, the codes were inductively derived at on the learning points on the use of big data. The first author then classified into dimensions. The second and third authors then reviewed the codes, which through extensive discussions, led to a resolution of differences picked up. Given the different cadres of the participants, views offered, for example a politician, were assessed against those offered by, for example a programme manager, given their differences in the organizational cadre. Inconceivable that their interests, and ultimately, the narrative provided could differ depending on the audience. This is consistent with Boje’s (2001) view that narratives could be selectively different depending on the audience. That is why elite participants had to consist of organizational actors from different top levels of the organization.

3.7 Findings

The analysis from the two case studies produced rich and insightful information on managerial use of big data for smart city initiatives. The overall statement that emerged from the cross-case comparison was that, apart from the learning loops that the city councils went through. The challenges that each city faced (contexts), the structure of the organizations, the people, the resources available, the initial results from initial smart city endeavours led to each city council adopting their own approach to creating a smart city. Given that these concepts are relatively new and fast-moving there were certain aspect of the initiative that proved to be a learning curve for each city authority. The findings centre on the following set of moves: (a) knowledge finding by embracing a smart city/big data approach for problem-solving (Top to bottom); (b) knowledge framing; (c) inter-organization collaborations (as well as through knowledge sharing); (d) ex-post evaluations from the smart city demonstrators.

3.8 Knowledge finding

By embarking on a smart city initiative the city councils have been able to reframe urban issues by searching for new knowledge from big data. Before the advent of big data, there was no systematic way of unearthing the knowledge around urban issues. The interviews reveal that the urban issues were faced from mainly two perspectives: political and organizational. The beginning of this learning process emerged, primarily, due to the emergence of available resources: financing. Thus it seems that the availability of resources made it feasible for the leaders/decision-makers to expedite the process. The excerpts from the transcripts illustrate the top-to-bottom approach.

So what we’ve been doing… So you’ve got technology, you find out whether it works for you or not. If it is going to be able to give you what you think it might give you. (C2.R1)

I don’t think we would have adapted as quickly as we had as a result of the concept, as a result of the [grant], I’ll be honest with you, in terms of the [grant] that was awarded to [Case 2]. I think we would have struggled, I think we would have struggled to get our heads round what it meant and how we could use it for the benefit of the city. I don’t think that’s [an] unfair criticism. (C2.R7)

I know, absolutely, I guess what are the critical success factors for making some of that (smart cities) happen? So of that might be about, OK, the resource to enable it to happen but also the kind of division. So if the division is there; but also the leadership to want to make it happen and, I think probably, there was a sense that all fell into place as a consequence of the grant (C2.R7)

The above narratives demonstrate that organizational resources were central to reframing problems. Despite the impact of resources in adopting a smart city approach and how the leadership can be carried along on this. Cities, themselves, have to possess a level of organizational competence to be able to attract such grants in the first place. This is consistent with findings from Murray and Donegan (2003), whereby in these cases the cities have created a learning environment. More specifically, cities have to be able to reinvent themselves either by drawing on its history or demonstrating its managerial competencies. There was a common occurrence within the two case studies, whereby through the use of big data cities have been able to broaden the boundaries their knowledge of the problems they face. The broadening of knowledge around problems also enables stakeholders to unearth hidden issues and links with previously unrelated intertwined problems. Most importantly, it gives organizations the opportunity to reflect on its past, which is pivotal to how it charts its future. The findings in the study are parallel to intuiting in the 4I model, which indicates evidence of exploitation and exploration.

[Case 2]… got very rich at that point but still huge poverty in the city, I mean unbelievable poverty in–in Victorian times. emm…but a–a a rich city then of course we lost everything in just under 20 years: ships, engineering, a lot. So we didn’t manufacture the–the we did. So we had to change and part of the attraction is that [we have] always changed with times. Maybe because emm… and I say this quite proudly, because there’s nothing else to do, it’s kind of desperate measures brings about creativity. So we had a renaissance in the 1980s and 90s, which was first of all cultural and sporting. As we go on its going into fields of finance and business, all the emm…services, tourism and events but crucially now, low carbon energy, engineering and design and one of the most crucial of all, life sciences. (C2.R3)

[Case 1] There is a long history of … being a sustainable exemplar going back many years. And there are lots of–of emm… sustainability organisations have their headquarters there because of that… [we have] has set for itself carbon targets. Most cities in Europe talk of 20% reduction by 2020. [Case 1] has set a target for itself of 40% by 2020. Which means we need to focus very clearly on how we could achieve that and the vision is that we could achieve that level of reduction and improve the sustainability of the city by using digital technologies. (C1.R1)

The organizational competencies are particularly important in relation to the ability to reinvent. The city’s and organizational history is, in turn, an asset to its ability to reinvent to itself. The above excerpts indicate that external factors also play a crucial role in precipitating reinvention. For instance, the UK government is currently embarking on measures in order to balance the nation’s finances (O’Hara 2015), which has inevitably led to cuts to the way public services are financed. The situation has led cities to find innovative ways to provide cost effective and efficient services.

3.9 Knowledge framing

Following the advent of using big data to address urban issues, cities have learned to identify problems in a new light—by framing the issues in the light of the advent of big data. Knowledge framing was facilitated through the conscious interpretation of the value big data offers to stakeholders. In other words, cities are now able to tackle problems in a different way, as opposed to the pre-big data era. Most decisions that were made seemed to have been based on intuition or political expediency, but by adopting data-driven approaches, cities can innovate, by re-identifying problems. In other words, big data allows cities to reinterpret their problems a need-basis.

… so actually what felt like a kind of intuitive response which was that we need to take more jobs to the people, the data didn’t support the work what so ever. So sometimes it’s challenging your own assumptions. We now have to work through that by looking at what the other potential solutions might be and what the data is telling us that we can formulate solutions that could be different from what we initially thought. Without that dataset, to be honest, we would have gone ahead and said that we were going to put all our money… But that’s not the right solution. (R4)

In demonstrating part of the smart city initiative in Case 1, R2 noted that:

… that information is being published in real-time. So someone can just spot there’s a problem. But, it’s more difficult to work out what you do because if you lose a key asset people are still going to get home. But clearly, you’ll like to say to people spread your journey, avoid this route, use public (not really), use a different mode, maybe rail would be better so you can avoid it. So that is good to tell what’s happening in the city: graphical. (C1.R2)

Reframing problems also comes with challenges, especially managing expectations of stakeholders: this can occur within and outside the city councils. On the challenges on meeting stakeholders’ needs, especially based on the fact that decision-making is increasingly data-driven in these organizations.

The other thing is managing expectations, isn’t it? Because if everybody is going to get interactive and they’re going to tell us what they think should be done, you know. People think, and they are wrong, trying to tell them and you sound like a dictator. Consultation does not mean everybody gets what they want… So managing expectation that big-is one of the hardest things from a politician’s point-of-view. We got all these happening now [open and big data for a smarter city], that’s fine; [For example] I’ll get my bin empty every week instead of a fortnight. But we have to say, we’re consulting on what–what you think we should be doing. And we have to take all these in and digest and analyze it. You may not actually get what you want but you will certainly get, with our support, a better service than you will get than now. (C2.R3)

The above excerpt is critical to how stakeholders reframe problems. The problem particularly centres on managing the wider expectations of other stakeholders—the citizens. Given that both cities could be said to be in the same category—with regards to championing smart city solutions—they do share experiences between each other. Within this study, during the course of the interviews on the aspect on managing expectations during this learning process, the following was picked up with regards opening up data publicly. Interviews with participants from Case 1 had learnt from the experiences of Case 2:

So the example that [Case 1] gave was that it had to do with the deprived wards and they published neutral data that says index of deprivation. But they met with the councillor who came at the bottom and said that you can’t publish [saying] my ward is the most deprived in the city. And sometimes politicians don’t like the truth, especially if does not tell the right story. (C1.R2)

The interpretations are also indicative of how language plays a pivotal role in enabling stakeholders develop their cognitive and strategic maps. These findings are also consistent with a proposition from Crossan et al. (1999) that specifically enables stakeholders develop a shared meaning what problems the organizations face. This shared meaning, as the next set of findings reveal, were lacking in exchanges with key private sector stakeholders.

3.10 Collaboration with partner organisations

The experiences from working with big data to address urban issues spanned across working with other organizations, mainly in the private sector; mainly because city authorities realise that there are certain competencies they do lack with regards to understanding the machinations of big data. To this end, outside collaborations were sought for through collaborations with non-profit organizations, private sector firms and academia. Both cities partner with universities to come up with smart cities solutions. The cities in this research have both built an innovation hub in their respective areas by partnering with universities in their environs. Cities that are in the same group—those have similar interests and experiences in smart cities—tend form working groups were they share ideas, as evident in responses from C1.R1 and C2.R5. However, a more critical learning point comes from the experiences of public sector organizations seeking to collaborate with big multinationals companies (MNCs).

In the early days of the program… we were absolutely intimidated with the language that was being spoken and some of the businesses that were coming in gave me a really hard sell. [Be]cause we didn’t understand it but it was like, I don’t know, they were trying to sell this like the next big thing and you didn’t know if it was the correct thing… It was really interesting and I think as we had the experience of working through this, we have come to the conclusion that a lot of people talk a great game but don’t necessarily deliver on and don’t necessarily understand the full capability of what it can do… it was it was hysterical at one point, we were just thinking, do we actually understand what language this people are talking. (C2.R7)

… the whole point of this infrastructure is that it needs to be more open and integrated. So for business that want to invest in something they want to control it, manage it, lock it down and make proprietary and not say this open, anyone can use it, anyone can interact with it… so I think it’s just some sought of real clashes of culture. But you do get the sense those things it’s going to be good when it gets done. (C1. R1)

The above excerpts reflect the dynamics of political action and, ultimately, clashes of power. With the benefit of hindsight, the experience of public sector organizations indicate that the city council felt more comfortable dealing with smaller or medium sized technology firms because they were flexible and equally ready to learn, given the smart city market is still, considerably, a new market. Even though it is not certain that smaller firms, as opposed to big MNCs know much more about smart cities, they come across as more approachable and less technocratic than big MNCs. Furthermore, collaborations are affected by the reluctance of private sector stakeholders to work in synergy; first because, it is perceived that if the private can own and manage what investment they make, they would not be interested in committing resources. Second, there is also a hangover from the dotcom bubble. C1.R1 is of the impression that given that many of the executives in technology firms have come of age from the dotcom era, when money was thrown at anything and everything that was internet-based which resulted in great losses for many those firms. Furthermore, this experience, in turn, exposes a tension between problem re-identification and collaboration moves. The relationship described between private and public organizations, is akin to interpreting and integration (feed forward) in the 4I model. Similar relationships are displayed in ex post lesson (feedback).

3.11 Ex post evaluations

Both case studies in this research are relatively new to the big data and smart city concepts, but are the forefront of using data to address urban issues. Also, the findings so far point to a top–down approach to institutionalization. Ultimately, both cities want to attain a more predictive, proactive and preventative capability. The cities want to attain preventative capabilities because it could help with early interventions through the development of civic innovation models. Furthermore, by opening data, citizens and other stakeholders would then be able to engage in the decision making process in the city. Thus, big data transforms how cities engage with the academic and private sector in order to address civic challenges; mainly because smart cities initiatives are designed around key policy areas such as healthcare, housing, transport and public safety etc. Therefore, by institutionalizing these routines from the demonstrations, the organizations are able to pave the way for patterns of interactions and communication to formalize the lessons learnt ex post.

We’ve got an opportunity to deploy new smart infrastructure by default, because we’re going to be rolling out new infrastructure. It gives [us] the opportunity given we’re investing to make investments in smart cities technology in this case. And interestingly focus on the civic innovations piece… First of all… unlock data from the very silos across the city, making data more accessible and when it gets more accessible how you create value from it, how you increase transparency, create more trust and empathy between public sector and communities… to help better decision-making, be that of increased access to data, so we can begin to correlate data interactions, which then gives insight into the inner workings of the city—be that in health, energy, transport or whatever. (C2.R6)

… new digital technologies such as sensors, monitors, actuators or autonomous vehicles or ultra-high-speed broadband or ubiquitous connectivity… all of these new technology, which are emerging or becoming more part of our lives can help a city council or any sort of authority to deliver their services run more efficiently better (C1.R1)

In relation to the 4I model, the ex post phase is akin to the tension evident in the intuiting and institutionalizing phases (feedback). In both cases, respondents highlighted that with regards other stakeholders in their respective organizations, there were difficulties in explaining the essence of adopting a smart city approach. However, as the schemes began to yield results, other organizational stakeholders bought into the program. Thus, the lessons learnt from embarking from a smart city approach has led cities to ask how they use big data to transform the way they engage with communities; specifically making open data more engaging. Still, what influences organizations in how they use big data in planning? One way would be to sign post services within communities, which will empower the communities (more stakeholders) to make local decisions. Such a move signals a decision-making process that adopts a bottom-up approach. Furthermore, on the backdrop on having engaged in a smart city demonstration, these cities are now able to build and embed competencies that can or serve as a foundation to embark on bolder smart city schemes.

The framework in Fig. 2 indicates that with the use of big data cities organisations are able to reframe their knowledge the problems faced in that bedevil their organisations. In doing so, organisations are able to re-identify problem in the light of the abundance of data at their disposal. Ultimately, as public sector organisations duly realise that they do not have the capabilities to carry out some needed tasks to address their problems, then deploy the services of private sector firms. Thus by working together on smart city initiatives, a learning eco-system is created which enables for there to be a continuous process of learning in the implementation and post evaluative phases of the initiative.

Fig. 2
figure 2

Conceptual model on organisational learning with big data

Full size image

4 Discussion

The advent of big data and its use for addressing urban issues—in the form of smart cities—reveals the dynamics and complexities of adopting these new and fast-moving concepts. The study particularly demonstrates an iterative (feedback and feed forward) process that forms a learning eco-system within organizations. With scant research available on the use of big data in (public) organizations, this research sets the groundwork for more exploratory research into the use of big data. Also, as city councils want to make their urban areas smarter, our foray into this study was informed by the literature on existing OL literature, especially those that had focused on public sector learning (Rashman et al. 2009). By using a smart city context, this study points out the learning points of using big data from a public sector perspective. The learning experiences from these public sector organizations unearths experiences public sector organizations have faced in trying to adopt a big data approach to problem-solving. The findings reveal that organizations that adopt the key learning points around the use of big data have the ability to use big data to reframe problems facing their cities; use data-driven mechanisms to build a knowledge base for problems facing their city; work collaboratively with partner organizations across various sectors and, most importantly, reflect on the learning phases they had gone through.

The conceptual model in Fig. 2 is an adaptation to the 4I model developed by Crossan et al. (1999); based on four broad actions: intuiting, interpreting, integrating and institutionalizing activities across the organizational levels. The difference, however, is that theirs cuts across all organizational levels but our adaptation does not. Our adaption is also different from past studies that have adopted the 4I model, such as Stevens & Dimitriadis (2004), which have been longitudinal and multilevel. Still, this research reveals that organizations can intuit, interpret, integrate and institutionalize knowledge. This is because by adopting the big data and smart city concepts start from the top levels of the organization: a top-down approach. However, as the cities in this study begin to open up its datasets, this could be reversed to become bottom-up, or could become hybrid. C1. R3 noted that opening data to the public will be disruptive to the decision making process for the city. Opening data to the public increases the amount of stakeholders directly involved in the decision making process. As a result, the major concern centres of managing the expectations of the increased number of stakeholders. C2. R3 mentioned, “consultation does not mean everybody gets what they want”. Getting this point across—on what to expect from the opening and accessibility of data—will pose significant challenges to public sector organizations.

Similar to the intuitive phase of the 4I model, there was a certain level of intuition in trying to understand problems that the city faced. The cases reveal that intuiting in public sector organizations is a conscious move. The cities played to their strengths in order embark on their smart city initiatives. Big data gave the city councils the opportunity to challenge the taken for granted assumptions on how things were always done in their respective organizations. The findings reveal that leadership was pivotal in there being a shift from the ways things had previously been done. In other words, the willingness to adopt a data-driven approach to tackling urban issues is subject to the availability of funds. Regardless, it was reported that even though money played an important role in adopting this new approach to city management, the cities in question had to prove to a large extent that they were worth those grants. In other words, they had to show that they possessed competencies that other cities did not have. This was widely attributed to having the leadership in the first place. This is consistent with Vera and Crossman’s (2004) strategic leadership influences elements of a learning system. Even though Vera & Crossman challenge the assumptions around transformational and transactional leadership, the bottom line, nevertheless, remains that leadership impacts the way an organization learns. In this study, these have been conditionally present in the way both organizations have adopted big data as a problem-solving tool.

The findings give a unique insight into the disruptive nature of knowledge reframing. This is the ‘sense making’ part of the process. As such, opening up data could lead to the unearthing of unforeseen problems, which the authorities might not be equipped to handle. Given the often political nature of the problems encountered, the issue of transparency could either make city leaders more alert or make them deflect from the real issues facing the city. Seo (2003) stressed that organizational defensive routines have their roots in political relations in an interventionist organizational context. If privacy is a major concern of big data to stakeholders (the wider public); managing expectations as a result of opening data is, in turn, the major concern to governments and organizations. Managing expectations is also akin to proposals made by Lawrence et al. (2005) who note that power and politics should be incorporated to the study organizational learning.

We also liken the collaborative phase of our framework to that of the integrative phase in Crossan et al’s (1999) 4I model. Public sector organizations are playing catch up with their private sector counterparts on the use of big data, at least in narrative. The findings reveal that integration leads to a friction and, indeed, a disconnection between private and public sector organizations. With regards to inter-organizational collaboration, the OL literature around this area acknowledges that there exist two inter-organizational learning processes from a public sector perspective: the recognition of novel knowledge and the need for inter-organizational learning (Rashman et al. 2009). However, in their assessment of factors that influence inter-organizational learning and knowledge transfer, technological disparities between partnering organizations is not considered. The observation in respect to Rashman et al. is also consistent with findings from Lane and Lubatkin (1998), who note that the ability of a firm to learn from another depends on their similarities. The study demonstrates that even though the experiences of public sector organizations could be different from those in the private sector, the knowledge transfer processes could be hindered considerably if respective differences are not acknowledged. The knowledge equilibrium would, however, be difficult to gauge given the different perspectives (and motives) different organizations might assume on adopting big data as an organizational tool.

The fourth phase of our conceptual model is analogous to the institutionalization phase in the 4I model. Ex post lessons become very important given the evolving and fast moving pace of big data/smart city concepts. However, at the point of conducting this research, it is unclear if, or how, institutionalizing would occur. So far, the literature indicates this to be next plausible move. It would, however, be too sudden to arrive at such a conclusion given the fast moving pace of the concepts that contextualize this research. The literature suggests that institutionalization occurs in the organization from the top to the bottom, opening up data to the public could reverse this, to become bottom-up.

By integrating knowledge finding with intuiting; knowledge reframing with interpretation; collaboration with integration and ex post lessons with institutionalization, indicates that in managing changes organization need to acquire and embed knowledge from other sectors. Already, the 4I offers a unique alternative to a linear learning process that fall under predefined phases that are not suited for the development and provision of services (Steven and Dimitriadis 2004). Above all. The 4I model has proved to be a useful model to analyze big data as a driver for organizational change would need to be investigated. As such, studying such a phenomenon can be defined based on the need to hold back or transform the intuitive ways of doing things. In doing so, public sector organizations have to invest properly on the right skills and infrastructure that would enable them harness the full potentials of big data.

Our adapted model facilitates the learning process as described above. For example, the findings suggest that particular attention needs to be paid to how knowledge is accessed, exchanged and embedded. Given that the private sector has primarily been driving the agenda on the use of big data as an organizational tool. This paper sets the ground for future studies to examine the kind of interactions—formal and informal—that occur in the learning process in public sector organizations and provides for a number of practical implications.

The practical implications of the study highlight the need for the firm to have the capability to learn and in doing so, to implement components of a modern application of the 4I model. In order to adopt big data tools (never before used) to support the process there is a need for organisations to implement appropriate training programmes supported by appropriate data-driven processes and policies, especially to build confidence to experiment with smart city solutions. Such urban policies, require a fundamental shift and alignment with broader organizational policies requiring organizational cultural realignment and a culture which encourages and values reflection and continual learning, reframing knowledge and proactively collaborating with other organizations for co-creation.

5 Limitations and areas of future research

Having engaged with the academic debate on the organizational use of big data, this research has demonstrated the need for organizational learning in public sector organizations. Thus, by exploring organizational learning through the lens of the 4I model, we were able to identify a set of learning moves pivotal to the learning process around the use of big data. The two case studies (smart city initiatives) were particularly useful in exploring and contextualizing the learning process and eco-system public sector organizations find themselves in. Following in-depth semi-structured interview with elite organizational stakeholders and ethnographic observations—supported by informal discussions and secondary documents—we were able to examine what and how public sector organizations have learned to use big data.

There are several limitations of this research. First, the primary data was derived from ten participants. However, given that big data is still very much at its infancy, especially in public sector organizations; this restricts the amount of stakeholders directly involved in the day-to-day dealings of big data. In order to mitigate this, secondary data—related documents—were collated to supplement this shortcoming. Furthermore, the informal discussions were particularly useful in triangulating on findings from the other sources of data as well as contributing to our knowledge of the use of big data.

Second, this research was conducted on a single level of analysis. At the time of conducting research, the infusion of the use of big data came from the top of the organization. Argote and Miron-Spektor (2011) note that because organizational learning occurs over time, there is a need for longitudinal and multilevel studies (Crossan et al. 1995, 1999; Marsick and Watkins 1994). Thus, since learning starts an individual level, these elements are then incorporated in groups; and what is learned is then transferred to the organization, which results as a change in the firm’s schema (Argyris 2004; Argyris and Schön 1978; Cohen and Levinthal 1990; Crossan et al. 1995, 1999; Nonaka and Takeuchi 1995). Another limitation is that evidence gathered for this study is restricted to smart city initiatives situated in one country. Hence, a multilevel longitudinal study would unearth far-reaching conclusions on the impacts of big data as an organizational tool.

Although this is not a longitudinal study, the alignment of the findings in this study to that of the 4I model indicates that knowledge on novel organizational tools such as big data can be explored with existing management theories and concepts. As demonstrated in this study, new theory emerging from such a tool would have to adapt itself to the fast moving developments of big data. The result of the present study sheds new light on use and adoption of big data in organizational tool. This research is important as organizations are increasingly using big data, internally—to quantify (capture, collect and predict) activities as well as to innovate around the provision of products and services.