IoT-enabled smart cities: a hybrid systematic analysis of key research areas, challenges, and recommendations for future direction

Abstract

Cities are expected to face daunting challenges due to the increasing population in the near future, putting immense strain on urban resources and infrastructures. In recent years, numerous studies have been developed to investigate different aspects of implementing IoT in the context of smart cities. This has led the current body of literature to become fairly fragmented. Correspondingly, this study adopts a hybrid literature review technique consisting of bibliometric analysis, text-mining analysis, and content analysis to systematically analyse the literature connected to IoT-enabled smart cities (IESCs). As a result, 843 publications were selected for detailed examination between 2010 to 2022. The findings identified four research areas in IESCs that received the highest attention and constituted the conceptual structure of the field. These include (i) data analysis, (ii) network and communication management and technologies, (iii) security and privacy management, and (iv) data collection. Further, the current body of knowledge related to these areas was critically analysed. The review singled out seven major challenges associated with the implementation of IESCs that should be addressed by future studies, including energy consumption and environmental issues, data analysis, issues of privacy and security, interoperability, ethical issues, scalability and adaptability as well as the incorporation of IoT systems into future development plans of cities. Finally, the study revealed some recommendations for those interconnected challenges in implementing IESCs and effective integrations within policies to support net-zero futures.

1 Introduction

Cities are a critical constituent of modern civilisation due to their environmental and socio-economic impacts on citizens’ lives [1,2,3]. Based on a recent report issued by the United Nations (UN), 55% of the world’s population currently lives in cities. However, the projections made by the UN shed light on the possibility of 6.5 billion people living in urban areas by 2050, equivalent to 68% of the world population [4]. This is triggered by urbanisation, a gradual shift in the residence paradigm of the human population from rural to urban areas, in tandem with the overall increase in the global population. As such, cities are expected to face daunting challenges since their resources and infrastructures are predicted to undergo an ever-increasing strain in the impending future [1]. In response, the concept of smart cities has emerged strongly over the recent decades owing to its potential for tackling these challenges through the deployment of Information and Communication Technology (ICT). Many cities around the globe have invested in becoming “smart”, aiming to improve city operations and the quality of services provided for citizens and the environment [2, 5] (Table 1). In a comprehensive definition presented by Kondepudi et al. [6], smart cities are characterised as cities that utilise ICT and other advanced technologies to increase the quality of life for citizens, promote competitiveness, and improve the efficacy of urban services while assuring the perseverance of resources for present and future generations. In this regard, technologies such as the Internet of Things (IoT) play a crucial role in enabling cities to transition toward the smart city paradigm. IoT can be defined as a global infrastructure offering advanced services by interconnecting various physical and virtual “things” using interoperable ICTs [5]. The employment of IoT in the built environment enables devices to communicate with each other using different methods, such as ubiquitous and pervasive computing, sensor networks, and embedded devices [7,8,9]. The concept of integration was initially introduced as smart city testbeds that offered a platform for researchers to investigate new methods before implementing them as robust solutions. Two large projects SmartSantander and OrganiCity were introduced as smart city testbeds using IoT experimentations at an urban scale in Europe [10,11,12].

Table 1 Examples of smart city projects in different countries

In recent years, increasing attention has been given to the deployment of IoT technologies to support smart cities to meet specific goals within Sustainable Development Goals (SDGs) such as Good Health and Wellbeing (SDG3), Industry Innovation and Infrastructure (SDG9), Sustainable Cities and Communities (SDG11) and Responsible Consumption and Production (SDG12). For instance, several applications of IoT in smart cities that are in line with SDGs including smart buildings, smart energy management, smart water management, health monitoring, environmental monitoring, intelligent traffic management, smart parking solutions, connected public transportation, smart waste management, public safety and surveillance. While emerging IoT technologies significantly contribute to smart cities aligning with SDG9, they also have an impact on the global economy. In this context, statistical information from several organisations such as ‘IoT Analytics’ on the global IoT enterprise spending dashboard indicated that the IoT enterprise market size steadily increased at a compound annual growth rate (CAGR) of 14% in 2019 to 22% in 2023 [18]. Allied Market Research [19] stated that smart cities and applications based on IoT are expected to reach $5.4 Trillion in 2030. The market was valued at $648.36 billion in 2020 and is projected to reach $6,061.00 billion by 2030 [19]. Statista indicated that approximately 50 billion IoT devices will be used around the world by 2030 [20], which has influenced the development of smart cities to increase from 118 cities in 2021 to 141 cities in 2023, as reflected in a report by IMD World Competitive Centre (WCC) [21].

Since 2010, various studies discussed the concept of the Internet of Things for smart cities. Initial database searching was conducted using the Web of Science (WoS) with the keywords “Internet of Things”, “IoT”, “and/for”, and “Smart Cities”. The search pointed out that the number of publications between 2010 and 2013 was limited. Subsequently, the number of publications in this field has increased, recording more than 100 publications in 2014 while it reached more than 1000 in 2022 in different fields. By targeting “highly cited papers”, the search returned less than 200 papers from the Web of Science Core Collection in different fields. Interestingly, the findings revealed three papers that received the highest number of citations above 1000, including studies by Zanella et al. [22], Botta et al. [23] and Lin et al. [24].

As mentioned earlier, a plethora of research has been developed investigating various aspects of IoT-enabled smart cities (IESCs). In response to the increasing number of publications in the field of IESCs, many review articles have been published aiming to solidify the flourishing knowledge in the field. The focus of these papers has mainly been limited to particular aspects of IoT in smart cities. Amongst them are studies that provided an overview of IESCs’ concept [3, 7, 9, 25, 26], studies that investigated the key IoT technologies and infrastructures for smart cities [8, 27, 28], and those that reviewed key features and applications of the IoT technologies to support the development of smart cities [2, 29].

Nonetheless, the rapid advancements in the field are outstripping the possibility of addressing various aspects of IESCs in a single literature review article, and this most likely can be the main reason for the absence of a comprehensive literature review in this field. In addition, performing a holistic literature review of IESCs can be challenging due to the multi-faceted nature of this research area in which the current body of literature often spans across multiple disciplines [5]. This may further point out the diverse, yet fairly fragmented intellectual base of IESCs.

Therefore, this study adopts a hybrid literature review technique consisting of bibliometric analysis, text-mining analysis, and content analysis to systematically analyse the literature connected to IESCs. To the best of the authors’ knowledge, this is the first study of its kind that investigates the IESCs literature using such a comprehensive review approach. The objectives of this study can be summarised as (i) to identify the key research topics in the field of IESCs, (ii) to critically analyse the most popular realms of IESCs research identified via bibliometric analysis, and (iii) to provide recommendations for future development of IESCs. The outcomes of this research offer a status-quo understanding of IESCs literature to the interested communities, providing them with a view of the most popular research streams as well as emerging research themes in the field. This can be particularly useful for the scientific community as the findings of this study shall furnish them with an understanding of research areas that require further investigations.

2 Methodology

The overall methodological approach of this research consists of three major stages, as illustrated in Fig. 1. The following sections provide further details on each of these stages.

Fig. 1

The overall research approach of this research

2.1 Database development

The choice of a database for performing scientific reviews is utterly important due to its impact on the quality of results [30, 31]. To date, several databases have been developed to assist scholars with conducting advanced searches through various bibliometric sources such as Medline, Google Scholar, ScienceDirect, Scopus, and Web of Science (WoS). The difference between these databases resides in their coverage when it comes to research disciplines [31]. Among all, the WoS is one of the most widely utilised databases for the purpose of review analysis owing to its distinguished features in enabling researchers to gain access to more than 171 million scholarly records available via 34,000 journals, allowing users to carry out advanced searches, and offering access to over 1.9 billion cited references across various disciplines [32, 33]. As such, this paper has selected WoS as the primary database for the retrieval of publication materials owing to its comprehensive coverage and scientific soundness.

The first step involved constructing a comprehensive search syntax consisting of terms related to the concept of IoT-enabled smart cities. To this end, a search string was formulated using keywords such as “Internet of Things” OR “IoT”. These keywords were thence combined with “Smart Cit*” OR “Cit*” OR “Urban” OR “Built Environment” via Booleans (“AND”) and deployed as the search query for retrieval of relevant data in the WoS database. It is also noteworthy to mention that the scope of the current paper is limited to the investigation of IoT applications in the context of urban environments.

The constructed search string was applied in the Web of Science Core Collection (including Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Emerging Sources Citation Index (ESCI), Conference Proceedings Citation Index-Social Science & Humanities (CPCI-SSH), Conference Proceedings Citation Index-Science (CPCI-S), and Arts & Humanities Citation Index (A&HCI)) database indexed since 1900 using the “titles, abstracts, and keywords” of scholarly materials. The search returned 4223 documents on the 2nd of October 2022 including 2018 articles, 1811 proceeding papers, 203 review articles, 81 early access, 66 book chapters, 36 editorial materials, 2 books, 2 datasets, and 4 miscellaneous.

Further, this paper considered several inclusion criteria to filter out materials irrelevant to the defined objectives. First, the “Document Types” filter was used to retain only documents classified as “articles”, “review articles”, “books” and “book chapters” since these materials are considered “certified knowledge” because of their reputability and comprehensiveness [34]. Second, documents written in non-English languages were also excluded. Third, resources that were not related to the IESCs (e.g., law, medical science, agriculture, nursing, parasitology, and fisheries) were filtered out using the filtering functions of the WoS.

This ensured that only documents directly relevant to the concept of IESCs were retained for further analysis. Thereupon, a peer-review check was conducted to ensure that the selected articles underwent a rigorous peer-review process. This was done by cross-referencing the publication sources of the shortlisted articles with recognised databases and directories of peer-reviewed journals, such as Ulrich's Periodicals Directory or the Directory of Open Access Journals (DOAJ). This thorough verification process ensured that only articles from sources with established peer-review processes were included in the analysis. In addition, the titles, abstracts, and conclusions of shortlisted articles were scanned to ensure their alignment with the scope and objectives of this paper. In this stage, studies must have investigated the IoT implementation in the context of smart cities to be included in the final analysis. Hence, materials solely focused on IoT implementation or exploring IoT in unrelated contexts (e.g., manufacturing) were excluded. It is also noteworthy to mention that studies with incomplete information, both in terms of methodology and reporting of findings, were excluded.

As a result, the application of these filters led to a downsizing of the initial search results to 843 documents. These materials were then exported using the ‘Tab-delimited’ file format to be processed and analysed via VOSviewer.

2.2 Identification of key research themes

The keywords co-occurrence analysis is commonly utilised for mapping out the theoretical and empirical knowledge in research disciplines [30, 31]. The application of this method enables researchers to demonstrate the cumulative knowledge of the target literature, unveiling the conceptual and thematic structure of the research and identifying key areas within a given research domain [31]. In this approach, the calculation is done based on the frequency of co-occurring keywords in publications and their corresponding strength of associations. The current paper constructed the network of keywords co-occurrence via VOSviewer software using all keywords of selected studies, including “author’s keywords” and “keywords plus”, e.g., those indexed by publishing journals.

VOSviewer is one of the bibliometric tools being widely employed across many research disciplines to assess the literature, such as smart cities [35], innovation in the construction industry [36], or construction waste management [37]. This software offers a user-friendly interface, allowing researchers to develop, visualise, and explore the bibliometric nexus of various entities related to a given research area [31]. One of the attributes of this software relates to the function of data cleaning via using a thesaurus file [38], providing the possibility of polishing the dataset to enhance the accuracy of analyses by merging different variations that may exist for one term. Hence, a thesaurus file has been developed to ensure the precision of analyses (e.g., IoT, internet of things, and internet of thing refer to one term, thus they have been merged and represented as IoT). Further information regarding data analysis and science mapping can be found in the software manual [38].

Further, this study applied text mining analysis via VOSviewer software. This technique is aimed at extracting information from a massive corpus of documents in texts [39, 40]. The use of text mining enables researchers to adequately capture semantic structures and prevalent patterns of phrases that characterise a large amount of data in text format [39, 40]. In this paper, text mining analysis was employed using the term co-occurrence algorithm to analyse the concatenation of titles and abstracts of 843 publications. The adoption of this technique, in combination with keywords co-occurrence analysis facilitated identifying research areas with the highest interest that shape the conceptual structure of IESCs.

2.3 Content analysis

Upon identifying the major thematic research themes, the selected materials were imported into an Excel Spreadsheet in order to be classified based on the results of previous steps. To this end, the titles, abstracts, and conclusions of all 843 studies were thoroughly read to allocate each study to its corresponding research theme category. Thereafter, studies in each category were critically analysed. The results of these analyses are provided in Sect. 3.

3 Results and discussions

3.1 An overview of results

Figure 2a illustrates the overall publication trend per year, focusing on IoT implementation in smart cities. Starting from 2017, a significant surge in publications can be observed with 75 articles published in that year alone which is equivalent to nearly 60% of all materials published in the previous 6 years combined. This notable increase can be attributed to multiple factors, such as increasing investment in innovative technologies, rising awareness of IoT's role in addressing urban challenges, and advancements in IoT technology and infrastructure [35, 41]. The impact of policies aimed at stimulating innovative ICT-based solutions for enhancing city governance can also be highlighted as one of the key contributors to the growing popularity of this topic, as many countries have begun supporting smart city projects by adjusting their policies over recent years [42, 43]. The support provided by policies can potentially drive further research and innovation in the field. Furthermore, the publication trend has shown a consistent upward trajectory in recent years, with 53% of all materials (i.e., 445) published between 2020 and 2022. Considering this steady rise in publications across these periods, alongside the escalating interest in ICT-based technologies, the interest in IESCs is expected to continue growing in the foreseeable future.

Fig. 2

a Publication trend of IoT implementation in smart cities. Note that the limited number of documents identified for 2023 is attributed to the search that was conducted in late 2022. Hence, we expect an increase in publications for 2023, indicating an upward trend. b Top 10 journals with the highest publications

The findings also revealed that a total of 127 journals collectively published 843 articles spanning the years 2010 to 2022. Figure 2b illustrates the top ten prominent journals that have significantly contributed to the advancement of the field, accounting for nearly 51% of the published materials. Notably, IEEE Access emerged as the leading publisher, with 14% of the selected materials included in this review, followed by Sensors–MDPI (13%), IEEE Internet of Things Journal (8%), and Smart Cities–MDPI (4%). A common characteristic shared among these top journals is their multidisciplinary nature, covering a diverse array of topics related to "smart cities" and "IoT." Yet, there is a particular focus in the scope of these journals that resonates with the main research themes (i.e., data analysis, network and communication management and technologies, security and privacy management and data collection) identified in this research (See the next Section).

3.2 Identification of key research areas using bibliometric analysis

Figure 3 shows the results of this analysis performed for a minimum threshold of 25 keywords. In this figure, the frequency of co-occurring keywords in the target literature is represented by the sizes of the nodes while the strength of associations between keywords is represented by the thickness of the connecting links. The location of the nodes can also be an important point of reference, indicating that the keywords with proximity would most likely have strong relationships with each other.

Fig. 3

Illustration of keywords’ co-occurrence analysis

Thirty keywords with the highest values of co-occurrence, along with their respective link strength are shown in Table 2, indicating that these keywords have received high attention in the literature and are strongly associated with other keywords. In this regard, the high values of “IoT” and “smart cities” keywords were expected because they were included in the search string used for the retrieval of primary data. Nevertheless, these terms were kept in the analysis since their removal would have led to omitting other keywords linked to them. As shown in Fig. 3, the results of the keywords co-occurrence analysis identified five major clusters. The findings suggest that keywords such as “smart buildings”, “artificial intelligence”, “big data analysis”, and “challenges” from cluster 1; “IoT”, “urban areas”, and “energy efficiency” from cluster 2; “cloud computing”, “edge computing”, “energy consumption”, “communication technologies”, and “5G networks” from cluster 3; “blockchain technology”, “trust management”, and “authentication protocols” from cluster 4, and “machine learning”, “deep learning”, “time-series analysis”, and “intrusion detection” from cluster 5 are located at the proximity of the clusters’ boundaries, implying that these domains of research are cross-cutting with solid associations with different clusters.

Table 2 Thirty keywords with highest occurrences

Table 3 presents the results of the text-mining analysis, introducing four key research themes within the domain of IESCs that have attacked the highest interest in terms of publications. These areas include (i) data analysis approaches, (ii) network and communication management and technologies, (iii) security and privacy management, and (iv) data collection approaches. The outcomes of the text-mining analysis are, to a large extent, consistent with the results of the keywords co-occurrence analysis shown in Fig. 3 and Table 2.

Table 3 The key research themes in IESCs

The next step for this paper is to critically discuss these four areas. Such an analysis can provide the target audiences with a state-of-the-art understanding of the recent developments in IESCs and create a proper basis for future research to contemplate further innovations and advancements in the field.

3.3 Critical analysis of research themes

3.3.1 Data analysis approaches in IESCs

Data in smart cities is generated and analysed based on multiple relationships between various technological systems and their physical environments. IoT data plays a vital role in the success of any smart city [72, 73]. Accessibility to real data offers the capacity to instantly assess the performance of any entity within a smart city by continuously collecting and analysing data from numerous sectors (e.g., health care or transportation), and actively attending to any anomalies manifested in manufacturing products [20]. Choi [47] stated that the quality of services in a smart city relies on the type of generated data gathered from different sources, levels and scales. Therefore, data collected from IoT devices should be effectively processed and transformed into actionable insights to regulate the massive flow of information in any smart city [74]. IoT data can be generated from various sources, including (i) equipment data to demonstrate the status of the IoT devices which facilitate activities of predictive maintenance, (ii) submeter data to measure utility usage (e.g., information about water, and electricity), and (iii) environmental data to evaluate and sense temperature, humidity, air quality and movements. IoT data is often generated in discrete values representing facts or numbers that convey information including useful, irrelevant or redundant information, hence these data need to be processed and analysed in order to be meaningful [75].

Data analytics is an integral part of deploying IoT technologies in smart cities, enabling the assessment of datasets to retrieve meaningful outcomes [76]. These outcomes are thence presented using statistics, patterns and forms that help to establish effective  decision-making processes [45, 77]. Moreover, data analytics provides solutions to overcome the issues attributed to unstructured data, including controlling variant types and formats of generated data [78]. Figure 4 presents the connection between IoT data to demonstrate the flow of collecting data from sensors installed in IoT devices and the process of transferring data through an IoT gateway for it to be analysed [79] and how different data analytics processes can be employed for processing data in the context of IESCs. Data of IoT in smart cities undergoes different complexities that are undertaken based on specific values as listed below:

• Descriptive/time series analytics demonstrates time-based data and massive in-motion data sets to identify urgent situations, instant actions as well as associated trends and patterns [50, 80].

• Diagnostic analytics employs data mining and statistical analysis to detect latent relationships and patterns in data that are applied to uncover the causes of specific problems [49].

• Predictive analytics aims to predict future events by employing various statistical and machine learning algorithms to develop models that can be utilised for predictions about future events such as weather forecasting [81].

• Prescriptive analytics uses both descriptive and predictive analyses to recognise suitable actions based on a specific situation which is common with commercial IoT applications to reach better conclusions [76].

Fig. 4

Data process in IoT and different types of data analytics based on complexity and value in IESCs

To analyse IoT data, it is vital to recognise the nature of data prior to processing. In principle, IoT data is categorised into structured and unstructured. Structured data follows a specific model to define how the data is organised or represented. Data collected via IoT sensors often comes in structured values, especially if these sensors are used for environmental assessments such as air temperature, humidity, or air quality. This type of data is simply formatted, queried, stored and processed. Unstructured data cannot fit into predefined data models such as text, speech, images, and videos, which requires conversion into a logical schema for decoding data. According to International Data Corporation, approximately 80% of business data is unstructured [5]. Therefore, data analytics techniques are gaining considerable interest in IoT, especially in smart cities due to their capacity for processing unstructured data [46, 82].

Data generated via IoT devices can also be classified as Real Time and Non-Real Time or in motion versus at rest [83, 84]. Most data in IoT are in motion as they move across different networks until they reach their ultimate target. However, inactive data is considered data at rest that can be stored in different digital forms such as mobile devices, spreadsheets, or databases. Mohammadi et al. [85] highlighted IoT data with five features: high volume, high velocity, high variability, value and veracity. High-volume data is distinguished by its enormous quantity, which is constantly produced via many IoT devices. High velocity refers to data type generated at high speed by many IoT devices. High variability data, which is inconsistent due to the dynamic nature of IoT environments, encompasses a variety of data formats such as unstructured, semi-structured, quasi-structured and structured data. Value data exemplifies the significance of collected data from IoT sensors after being analysed. Finally, veracity data promotes consistency, quality, and reliability in generated data.

Yang and Shami [45] and Yin et al. [51] demonstrated tasks in IoT data analytics that consist of classification, clustering, regression and anomaly detection. Furthermore, Yang and Shami [86] indicated that information from IoT is processed according to the type of data. Alternatively, algorithms in IoT data analytics are categorised into two types of learning: batch and online. L’heureux et al. [87] indicated that batch learning represents a method to analyse static IoT data in batches via Traditional Machine learning algorithms. Online learning is a technique that employs various methods to train models in IoT environments by continuously using incoming online IoT data streams [88].

Data analytics relies on advanced methods to perform analysis. Among all, machine learning (ML) has gained momentum for providing new knowledge and improving data quality via learning and processing repetitive data to attain efficiency [89]. ML employs two forms of techniques, namely supervised and unsupervised learning [90]. The machine learns rules and models of datasets drawn from clustering or frequency of particular data using pattern recognition and reinforcement techniques [90, 91].

Another analytical technique often used for data processing in IoT-enabled platforms is deep learning. Li et al. [92] described deep learning as an ideal method for obtaining precise information from raw IoT data that existed in complex environments. Atitallah et al. [76] categorised deep learning into different modes of learning such as unsupervised, semi-supervised, supervised and reinforcement. Deng [93] classified deep learning into discriminative, generative and hybrid classes. According to Atitallah et al. [94], the most common deep learning models are Convolutional Neural Network (CNN), Deep Belief Network (DBN), Deep Reinforcement Learning (DRL), Generative Adversarial Networks (GAN), Recurrent Neural Network (RNN) and Stacked Auto Encoder (SAE). Figure 5 summarises different deep learning models, along with their applications. IoT data analytics utilises different advanced computational platforms to improve performance and accuracy. Three types of computational analysis cover computing using Cloud, Fog and Edge [94]. In fact, both Fog and Edge represent an extended development of cloud computing that offers the power for data analytics to be performed near the source of generated data.

Fig. 5

Taxonomy of deep learning models and their main applications

3.3.1.1 Applications of IoT data analysis in IESCs

The process of retrieving information from data generated via IoT devices requires employing data mining tools. Mining data is a process of identifying patterns and correlations or discovering anomalies within large datasets to forecast outcomes. Various types of algorithms could be used for data assessment such as clustering algorithms. The selection of algorithms for clustering data is based on considering different variables such as size, data size, data type and the number of clusters. Daissaoui et al. [95] listed several algorithms to generate and manage data in smart buildings. The study covered four general types identified in previous studies including probabilistic graphical models, system identification methods, vector support machines and data mining and clustering. Several studies used probabilistic graphical models, e.g. Stoppel and Leite [96] incorporated probabilistic methods into simulation by analysing building energy models for describing occupant presence in buildings. Chen et al. [97] proposed methods using stochastic inhomogeneous Markov chains to examine occupancy in single-zone and multi-zones within a building. For vector support machines, Akbar et al. [98] investigated the occupancy state in a smart office by proposing a non-intrusive approach via Support Vector Machines to detect an occupancy state by using electricity consumption data. For data mining and clustering, D’Oca and Hong [99] presented a framework that utilised a three-step data mining through a decision tree model. The proposed model was used for forecasting occupant presence and occupancy patterns in office spaces.

Hong [100] suggested an approach to operate data analytics using Fog and Edge computing and central servers to enhance decision-making from IoT devices. Yang [101] and Rahman et al. [102] proposed a model to ingest IoT data into the nodes of Fog computing. The model covers the limitations relating to the fundamental aspects of data analytics associated with data, humans, systems and optimisation. Portelli and Anagnostopoulos [103] proposed a learning approach that enhances the prediction and precision of IoT data by using Adaptive Vector Quantization and Linear Regression to maximise communication efficiency. Lujic et al. [104] used a set of algorithms to examine IoT data that is affected by failures in sensors, systems and networks concerning smart buildings to recover incomplete datasets, reducing forecasting error and decreasing running time.

Several studies presented frameworks that combined data assessment from real-time and historical records to achieve prediction by using Decision Tree Regression, Multiple Linear Regression, Support Vector Regression and Random Forest Regression [105]. For instance, Xiaoyi et al. [48] presented a model for smart cities by analysing the management of renewable energy systems using a Multi-Objective Distributed Dispatching algorithm. The proposed method managed to decrease energy consumption while delivering high utility services in a smart city. Gomes et al. [50] also presented a framework and modules to facilitate data analytics in real-time and data stream enhancement. The analysis layer was developed to include a set of modules to extract relevant information as shown in Fig. 6. These modules include (1) pre-processing to eliminate invalid values and reject values out of a specified range; (2) aggregation to aggregate a data set by using various functions (e.g., min, max, sum, or count); (3) statistics of a dataset in the form of median, average, standard deviation, variance, kurtosis and skewness; (4) pattern to detect behaviour patterns such as trend changing, and stability; (5) clustering to group datasets based on distance or similarities using density-based clustering, hierarchical clustering, k-means, or subspace clustering, and (6) prediction to forecast values in several steps using various approaches such as Autoregressive Integrated Moving Average, Artificial Neural Network (ANN), Kalman filter, and Forecasting Method to Model Time Series Data. Table 4 summarises the main applications of data analysis research in IESCs.

Fig. 6

A model of data analysis in IoT processing layer

Table 4 Summary of various research approaches of data analysis in IESCs

The summary showed that data processing and management in IoT devices are considered critical. The survey demonstrated various algorithmic techniques to facilitate data. It was found that studies are still exploring numerous methods based on the level of processing. The assessment revealed that many studies have been developed to create actionable platforms to enhance the data stream and data management, control data anomalies and improve data analytics. The assessment indicated that processing real data from IoT is challenging due to limitations in IoT architecture, data speed, size, accuracy, response and security. It was noted that many studies are still reviewing and testing the integration of different algorithmic techniques for new applications to obtain effective solutions. In addition, developing new IoT frameworks by integrating different algorithmic techniques has added a new level of complexity in designing smart cities. On the other hand, these findings indicate that IoT and big data analytics are still under development and require further research and investigation, especially in designing smart cities.

3.3.2 Network and communication management and technologies in IESCs

IoT network consists of various components such as sensors, software, gadgets and appliances that communicate and exchange information data with each other. Management of IoT networks allows for various functionalities such as authentication, configuration, provisioning, routing, monitoring and security to maintain a network performance in terms of low energy consumption and low latency [107]. Aboubakar et al. [108] stated that standard network management in IoT consists of specific logical elements that include agents, network devices, managed devices, and network managers which are supported by management databases and messaging protocols. Each element performs specific tasks to keep the network running. The ‘‘agent” represents the software which operates on managed devices or groups of IoT devices. The agent performs data aggregation into a combined stream to central IoT applications and is typically managed by IoT Gateway. The “network devices” include Firewalls for a security feature, Servers to manage the devices within that network, Client Applications to allow users access to complete tasks, Routers to connect to networks, Switches to allow the devices to communicate with each other and Access Points to connect the endpoint device with the network. The “managed devices” allow organisations to better monitor and control their connected IoT devices to register and deploy connected devices, device logging, organising devices into relevant groups, indexing and searching device fleets, remotely manage and update devices, custom scripting, security tunnelling for diagnosing and resolving issues and customisable dashboards for centralised device control. The management of devices plays an important role in increasing the speed of registration of IoT devices, improving device organisation, easier remote-device management and simplifying device location. The “network manager” is a device that manages a group of managed nodes. It facilitates network topology, synchronisation of devices, and management of traffic and congestion in IoT systems. The ‘‘management database” is located in the managed device and includes data about the managed device parameters. The ‘‘messaging protocols” function as a data exchange that connects information between the managed devices and the network manager. Generally, these networks need to be efficient to support specific functional operations (Fig. 7), such as Network Configuration Management, Security Management, Topology Management, QoS Management, Network Maintenance and Fault Management. These functionalities are typically provided as a network service in an IoT environment to ensure sufficient network performance.

Fig. 7

Overview of entities, operations and solutions in IESCs network management

Aboubakar et al. [108] presented management solutions for IoT networks from various perspectives. These IoT network management solutions included software-defined networking (SDN)-based, machine learning-based, cloud-based, and semantic-based frameworks. The design of a new network must incorporate efficient management processes for managing a significant number of linked devices, immense amounts of data, and services with varying Quality of Service (QoS) requirements. Monitoring the network’s infrastructure makes it possible to detect any events or changes that might impact the network's resource security or usage. In this regard, several protocols have been developed to help with network management. These protocols control and monitor different network components such as gateways, devices, and terminal servers. Since the implementation of IoT low-power networks in both public and private spaces is growing at unprecedented rates, network management has emerged as a critical component of IoT low-power networks for maximising their performance and ensuring their continued availability.

3.3.2.1 Communication technologies in IESCs

In computer networks, there are several common types of network technologies such as Local Area Network (LAN), Wireless Local Area Network (WLAN), Virtual Private Network (VPN) and Wide Area Network (WAN). LAN connects devices in the same proximity, e.g., connecting devices in a small office or a building. WLAN functions in the same way as a LAN, but it uses wireless connections. VPN is a secure network which is used to communicate with encrypted data. WAN offers the possibility to connect devices across a large distance. In smart cities, heterogeneous objects are connected by IoT communication technologies to deliver intelligent services. In IoT, several wireless network types can facilitate IoT sensor deployment and IoT applications in industries such as Radio-frequency identification (RFID)/Bluetooth Low Energy (BLE)/Near Field Communication (NFC), Wi-Fi/(LoRa and Wi-Fi)LoFI, MESH Protocols, NarrowBand-Internet of Things (NB-IoT), Ultra-Wide Bandwidth (UWB), Low-Power Wide-Area Networks (LPWAN) (LoRa, Sigfox), ZigBee and Cellular (3G/4G/5G/6G) [109]. The next paragraph highlights some of these technologies.

Bluetooth is a short-wavelength radio-based communication standard for low-power data transfer between electronic devices over short distances [110]. Bluetooth 4.1, lately issued by the Bluetooth special interest group, offers Bluetooth Low Energy besides offering high-speed and IP connectivity to promote IoT applications [111]. Machine to Machine (M2M) is the next generation of the Internet revolution connecting many devices via the Internet. The M2M was initially implemented using RFID as the first technology (RFID tag and reader), and now it is aimed at automating the communications between machines and devices via provided networks without human intervention. Ultra-wideband communication (UWB) is a communication technology that was developed to strengthen communications in areas with a low-range coverage while consuming a low amount of energy and providing a high bandwidth. Recently, the number of applications using this technology to connect sensors has increased [112]. Wi-Fi is a wireless networking technology that allows devices within a 100-m radius to exchange data using radio waves [57]. In certain ad hoc configurations, Wi-Fi enables smart systems to communicate and exchange data without requiring a router. For low-power wireless networks with the goals of reliable and scalable communications, the IEEE802.15.4 standard details both the medium access control and the physical layer [113]. LTE-A (LTE Advanced) represents an enhanced variant of Long-Term Evolution (LTE) that offers benefits such as increased bandwidth (up to 100 MHz), spatial multiplexing on both the downlink and the uplink, wider coverage, greater throughput, and lower latency [114].

There are a variety of technologies developed to improve the effectiveness of network management. One of these technologies is Radio Access Network as a Service (RANaaS) which has been developed to facilitate adaptable management of network resources [115]. IPv6-based networks have management protocols in place, such as Long-Range Wide Area Network (LoWPAN), Network Management Protocol (LNMP) and Simple Network Management Protocol (SNMP) [116]. In addition, self-organising wireless networks can benefit from Time Synchronised Mesh Protocol (TSMP), a communication protocol that empowers the sensors/devices to be synchronised with one another. Furthermore, Software-Defined Networking (SDN) is a key component in the development of 5G systems, which intends to reduce complexity in network management and design while also allowing for the network to be managed and reconfigured in a way that is automated, flexible and dynamic [117,118,119]. Furthermore, the paradigms provide features for managing heterogeneous devices in a wide range of deployments and use cases [120]. Despite the potential of these paradigms for introducing effective methods in managing networks, several issues remain [121].

Several studies such as Cedillo-Elias et al. [56] proposed cloud platforms utilising SDN and OpenStack to safeguard citizens' data contained by the government and make extra efficient usage of existing IT infrastructures for smart city facilities. In another study, Purnama et al. [54] analysed the viability of deploying IoT connectivity for AMI (Advanced Meter Infrastructure) services in Surabaya, Indonesia. To experimentally evaluate and compare multi-hop and single-hop LoRa topologies in terms of energy efficiency and range extension. Aslam et al. [122] presented a case study that measured Packet Reception Ratio (PRR) for different source-to-destination distances, transmission powers and spreading factors (SFs). The findings demonstrated that the configuration of a LoRa network with multiple hops can save a significant amount of energy and improve coverage. Nashiruddin and Nugraha [53] investigated LoRaWAN's network planning for Smart Metering Infrastructure (SMI) by counting the number of gateways required to support the communication of SMI devices. Fraile et al. [55] also compared IEEE 802.15.4 and LoRa for indoor deployments in IoT-enabled school buildings. Using information gathered from 8 networks and 49 devices spread across 6 educational facilities, the study compared the efficiency and cost of various IoT solutions. The outcomes demonstrated that LoRa can achieve lower costs and higher data rates than IEEE 802.15.4 while maintaining similar or better link quality. Table 5  presents the recent protocols to enhance the management of IoT communication.

Table 5 Protocols and solutions to enhance IoT network management

To sum up, this section discusses various network and communication management and technologies used in IESCs to connect heterogeneous objects and provide intelligent services while consuming low power. Intriguingly, surveying the literature on IoT network management shows no detailed or comprehensive overview available of existing resource-constrained network solutions. Future research could focus on developing more energy-efficient and scalable communication technologies to handle the increasing number of smart devices in smart cities. Further research should aim to investigate several solutions to improve data security, reliability, energy efficiency, network scalability, interoperability, and data privacy. These solutions include encryption, authentication, access control, low-power communication protocols, energy-efficient hardware, cloud-based architectures, edge computing technologies, blockchain, digital certificates, context-aware, adaptive IoT communication systems, and hierarchical clustering. Numerous studies have investigated the use of IoT connectivity for various applications, such as smart city services and advanced meter infrastructure, and have compared the efficiency and cost of different IoT solutions, including LoRa and IEEE 802.15.4. Therefore, it is recommended that IoT network management should incorporate efficient management processes for handling a large number of devices, vast amounts of data, and diverse services with varying QoS requirements. Furthermore, network managers should continuously evaluate and adopt new technologies and protocols to enhance network performance and security. Developing an effective solution for managing IoT networks can be challenging due to the inherent constraints of IoT networks [123], such as the diversity of IoT devices, the fluidity of network topologies, the scarcity of available resources, and the unpredictability of radio links. More research is needed to design effective solutions to manage IoT with low-power networks that can handle heterogeneity while ensuring security and privacy and allowing for scalable resource utilisation.

3.3.3 Security and privacy management in IESCs

Smart cities are equipped with advanced technological infrastructures to actively monitor and control physical objects and furnish citizens with real-time information about transport, smart parking, traffic, or public safety [124]. Nevertheless, there are various issues related to security and privacy at different levels of smart cities’ architecture. This is largely due to the nature of devices deployed in these cities which are often resource-constrained, thus making cities vulnerable to security attacks [64]. An example of such attacks is the major electricity breakdown that occurred in Ukraine due to malicious attacks on smart grids [124]. Therefore, this section discusses six major areas of security and privacy issues in IESCs.

Intrusion Detection System (IDS). As the number of things connected to systems increases, the centralised or cloud-based IDS will suffer from excessive latency and network overhead. Subsequently, it makes it difficult to respond to attacks and detect rogue users. For example, a fog-oriented IDS was developed with the capacity to use an Online Sequential Extreme Learning Machine, which is decentralised in computing infrastructure and has no fixed location between the data source and the cloud [125].

Automobiles and Transportation. Attempts have been previously undertaken to develop a “Smart Accident Precognition System (SAPS)” aiming to minimise the risks of accidents and protect the users’ safety on the road. To further improve the system, SAPS was coupled with Google Assistant to make use of various embedded devices for monitoring several aspects of vehicles and passengers such as speed, distance, and safety measures. The real-time data are stored within the cloud and accessed by both the vehicle and the Google Assistant, allowing smarter decision-making and acting based on the previous data recorded [67]. However, the implementation of such a scheme may pose threats to unauthorised access to users’ personal details as well as gaining control over vehicles and transportation systems.

e-Healthcare System. This is an improvement to the traditional healthcare systems by connecting to IoT systems and the Internet. However, e-Healthcare systems are subject to the same issues as any IoT-enabled systems such as compromising privacy and personal data due to hacking by malicious users from across the Internet [126]. For example, in a smart city environment, the healthcare system is often a collaboration between the public and private sectors. Although the public sector can be the central decision-maker, distributing treatment and medicine to the private sector may be more effective and efficient. However, this means the personal information will be overseen by different parties and thus have a larger chance of being hacked or exposed [127].

Communication methods of IoT. Since the communication of IESCs is reliant on online networking systems, it is susceptible to different types of cyberattacks [124]. In communication, connectivity is a critical component in delivering an IoT solution. Many protocols can be employed within the same IoT solution to maintain the stability of IoT communications to be suited for varied contexts with different barriers and limits. Some of these difficulties relate to the physical elements, i.e., the distance between devices, the specific IoT task performed such as the need for real-time applications requiring higher and more stable connectivity capabilities, and the device’s computing resources such as weaker or power-saving devices may need communication protocols that require less power. Every of these communication protocols has their strengths and weaknesses and some of them are more prone to attacks [127].

Code and program level of IESCs. Data aggregation has different levels, such as the need to achieve trust and quality in shared information models to enable reuse, secure data interchange and transfer, and protection mechanisms for vulnerable devices [59]. As each of the IoT solutions is deployed with different objectives and means, on the coding level, they should be customised and carefully integrated to maximise their performance and data protection. As an example, most websites and Internet-connected apps incorporate at least one type of Web API to assist with a specific function in the grand scheme of things, such as Samsung SmartThings, which provides classes via an API to process HTTPS calls within the IoT solution asynchronously [127].

Ethics and morality of humans. Caution is required with those working with the system and those holding the collected data, as there are research papers which showed that immoralities and irresponsible acts of corporations and authorities are important causes of compromised security and privacy. A study conducted in a smart city in India showed that although not all the usable subjects think that it is significant, in general, the trust and intention of the authorities and holders of the gathered data are affecting their trust in IoT as a whole [128].

3.3.3.1 Managing and combating the issues

To combat the issues highlighted above, active attempts must be made to improve security and privacy by safeguarding communications of devices and networks. To this end, specific measures can be implemented.

• (i) To combat the ineffectiveness of a centralised IDS, hybrid semi-distributed and distributed intrusion detection systems can be promoted. In these systems, the associated databases demonstrate effective feature extractions and selections, combined with parallel machine-learning models and fog-edge coordinated analytics that can mitigate the risks of centralised IDS [62].

• (ii) To combat possible security and privacy threats associated with transportation and automobile systems, including both unauthorized use of users’ information and attacks triggered by malware, spam, black holes, wormholes, and outages, it is necessary to improve Vehicular Sensor Networks in IoT-enabled transportation and automobile system in terms of robustness, reliability and security [65].

• (iii) To improve security and privacy issues, an active defensive line is needed consisting of ML and blockchain technologies in which the former enables predicting and detecting vulnerabilities while the latter secures the networks by creating tamper-resistant records of shared transactions [61].

• (iv) Securing different data layers is also a promising way to prevent security and privacy issues via using payload-based symmetric encryption for the data security layer, utilising computation of secured data for the data computational layer, and only extracting visions from the last data layer, i.e., the decision-making layer [60].

• (v) The physical layer is often ignored to be protected. The passive observer’s data is usually unreachable to the network's authentic source and destination nodes, deploying countermeasures such as an efficient “Sequential Convex Estimation Optimization” algorithm that can be very useful against them [129].

Table 6 presents a comprehensive summary of recent research in the field of security and privacy in smart environments. One of the notable observations is the diverse range of applications covered in the papers, including smart cities, e-healthcare systems, industrial IoT, smart homes, and more. This highlights the importance of context-specific solutions that can address the unique security and privacy challenges in different environments. The proposed solutions in the papers utilise various technologies, such as machine learning, blockchain, elliptic curve cryptography, and homomorphic encryption. While these technologies offer high levels of security and privacy, they also require significant computational resources, which can be a challenge in resource-constrained smart environments.

Table 6 Summary of recent research in security and privacy in smart environments

The limitations and trade-offs of the proposed solutions in the papers need to be carefully considered as well. For instance, Arunkumar et al. [130] presented a lightweight security key generating system that can detect and prevent security threats in smart cities using machine learning and elliptic curve cryptography. However, the hardware used to deploy the system is relatively immobile and consumes a lot of energy. The blockchain-based authentication method offers improved communication metrics and privacy-preserving features but comes at the cost of weaker identity management and slower automation speed [131]. Similarly, the routing model presented by Haseeb et al. [132] can efficiently establish direct trust between nodes but does not handle malicious attacks and flooding of messages well. The seamless authentication IoT framework for e-Healthcare systems presented by Deebak et al. [133] is more efficient and has a better packet delivery ratio and network lifetime, but consumes more resources compared to related works. Additionally, some papers address specific challenges in smart environments, such as predictive computation [134], security in fog computing [135], security in smart grid networks and access control in edge computing [136]. These papers highlight the importance of addressing specific security and privacy challenges in emerging technologies and infrastructures in smart environments.

In conclusion, Table 6 presents a summary of recent research in the field of security and privacy in smart environments. The diversity of applications, technologies, and challenges discussed in the papers highlights the need for context-specific solutions that can balance the trade-offs between security, privacy, efficiency, and scalability in resource-constrained smart environments.

3.3.4 Data collection approach in IESCs

The approach for data collection in the context of smart cities may vary depending on the smart data applications from macro to micro scales and the sectors from which the data is collected [138,139,140]. Different studies showed research in areas such as Energy Conservation, Urban Environment, Health & Wellbeing, Biodiversity, Surveillance/Security & Safety, Transportation & Mobility, Infrastructure & Communication, Tourism and Waste Management. Below, the review identifies a number of domains in which IoT can be integrated with advanced technologies for the purpose of data collection in smart cities.

• Electrical Energy. Several studies collected and forecasted data from the microgrid using smart meters such as the Heuristic Intelligent Neural Decision Support System [141,142,143]. In a study, Abu-Rayash and Dincer [144] developed a new integrated solar energy system capable of meeting the energy demands of a small city of 5000 homes. The proposed system can also collect real-time solar energy and thermal energy data using Photovoltaic panels and thermal energy storage tanks, respectively.

• Urban Environmental Pollution. Yu et al. [145] showed the effective collection of air pollution data (Particulate matter—PM2.5) from 242 cities in China based on an online IoT monitoring system. In another research, P.M2.5 sensors were deployed on street levels and via drones at Xidian University and Peking University using 4G (fourth-generation) internet network base platforms and the stations complied with narrowband IoT communications [146]. Furthermore, electrochemical (SNS-MQ135) and MQ9 gas sensors were employed via Bluetooth, ZigBee and Z-Wave networks to measure air quality at the polluted city Bucharest, Romania in order to detect carbon dioxide (CO2) level, ammonium (NH4), ethanol (C2H6O), toluene (C7H8), carbon monoxide (CO) and methane (CH4) [147]. Segura-Garcia et al. [148] also validated an IoT prototype for monitoring real-time Psycho-Acoustic Soundscape utilising 5G (fifth-generation) LTE-M1 sound monitoring devices. Further, Dembski et al. [149] developed an urban digital twin and computational simulations such as space syntax, SUMO—Simulation of Urban Mobility and simulated wind flow by installing a test sensor and mobile App – Reallabor Tracker for citizen’s feedback.

• Biodiversity. In a study, Chen and Han [150] used a range of turbidity, oxidation–reduction, or pH potential (ORP), conductivity and dissolved oxygen (DO) sensors to measure water quality in Bristol, UK. In this project, a multi-parameter water quality sonde (Aqua Troll 600) was used to assess the water quality while an IP Network-based Camera was utilised to collate video images of the water surface. Studies also used low-cost data collection methods through renewable wireless sensor networks for measuring environmental parameters such as temperature, pressure, humidity, smoke, and noise sensors, smart IoT-enabled bins, pyroelectric infrared, UV/Lux sensors and rain sensors [151]. For instance, Gallacher et al. [152] deployed Echo Boxes that consisted of low‐cost sensor networks combined with artificial intelligence techniques to monitor bats’ activities in a large urban park. Podder et al. [153] also proposed an IoT-based Smart AgroTech system in the context of urban farming with the capacity to monitor humidity, temperature, and soil moisture, and decide when the irrigation system should operate.

• Infrastructure, Information, Security and Safety. For example, real-time e-learning data collected via phones and gadgets can be applied in virtual classrooms [154]. Kinawy et al. [155] developed an online portal where the use of citizen profiles and knowledge items, such as tagging and comments on project websites were utilised. Similarly, business, parking, and tour information, i.e., users’ real-time scores and interaction, were shared with the people utilising e-government platforms and Mobile Apps at Petaling Jaya City Council and Putrajaya Corporation in Malaysia [156]. Recently, drones were also used for monitoring disasters, search and rescue tasks, surveillance and taking photographs with aerial views [157]. Shah et al. [106] developed a comprehensive disaster management model by which data can be collected from various sources such as Twitter datasets, weather sensors, surveillance sensors (e.g., CCTV cameras), pedestrian count, location, time, screen sensors for tracking vehicular traffic, and pollution and smoke sensors [106].

• Transportation. Recently, Sato et al. [66] proposed a prototype that included a crowd road surface sensing system on a sensing vehicle on a winter road and a sensor server system using an Axis Mechanical sensor, GPS, temperature, humidity, quasi-electric sensor and infrared laser. This IoT-based server is connected to a communication server. Chakroun et al. [158] proposed a system to reduce delays during emergency traffic by focusing on the density of vehicles vs delays in alert dissemination. The project incorporated the Location-based Alert Messages Dissemination Scheme and the sensors that are provided in the vehicle cluster system look at the speed and flow of traffic using cameras. Ajay et al. [159] proposed smart management systems using IoT sensors such as CCTV, fuzzy logic, pedestrian sensors, and ultrasonic sensors. Li et al. [160] analysed data from 8,900 personal cars for three months in the city of Changsha, China from an IoT-based vehicle monitoring system. Toutouh and Alba [161] developed a data collection method using broadcasted beacon frequency (Hz) to neighbouring vehicles to maintain traffic safety, congestion control and efficiency.

• Waste Management. Solid waste management with IoT was deployed using bin-level monitoring at home and public spaces using ultrasonic sensors and LoRaWAN networks [162]. Similarly, IoT-based sensors attached to bins included automatic open/close smart bin lids, filling level sensors, smart bin waste segregation, garbage collector alerts and ultra-sonic human detectors which helped arrange waste management systems and minimised delays in collecting bins when they are full [163]. Cerchecci et al. [164] proposed a prototype with an ultrasound distance sensor and a microcontroller for determining the level of bins and the count of changes in the bins.

Nonetheless, the process of data collection via IoT in the context of smart cities is being challenged by a number of factors (Table 7).

Table 7 Summary of challenges associated with IoT-based data collection methods

Based on the systematic review of surveyed articles, Table 7 illustrates data collection methods with a special focus on data collected from IoT devices, relevant human factors and other related systems. From the literature reviews in Table 7, it was observed that there is an extensive range of data collection methods applied in IESCs. Despite the collaborations between IoT sensors and other types of data sources, there is still a lack of consistency when the human factor is involved due to the unpredictability of data and variables. The bulkiness of IoT devices may be reduced in the future with the advancement of technology. However, privacy, security and reliability of human data are the major apprehensions in the successful implementation of smart cities. Finally, Fig. 8 illustrates the main data collection approaches applicable in different sectors of smart cities.

Fig. 8

IoT-based data collection approaches in various application sectors of smart cities

4 Challenges and recommendations

This review showed that IoT-based technologies have a critical role in realising smart cities. In the following sections, the key challenges and associated solutions are expanded by focusing on interconnectivity and integrating those challenges for more sustainable smart cities. From the critical analysis of the research themes, i.e. data analysis approaches, network and communication management and technologies, security and privacy management and data collection approach in IESCs, seven key challenges have been identified and these are elaborated below:

• Energy consumption and environmental issues: IoT offers the possibility of collecting, analysing, and delivering massive amounts of data via advanced communication technologies. The big data received from IoT devices requires storage capacity, cloud computing, and wide bandwidth for data transmission [174]. However, the entire process of analysing and transmitting big data can be very energy-consuming. This is in addition to the amount of energy that sensing devices consume to continually remain operational. Therefore, there is a concern about the energy efficiency of IoT implementations in smart cities to meet specific SDGs such as Sustainable Cities and Communities (SDG11) and Responsible Consumption and Production (SDG12). This is compounded by issues associated with e-waste generation due to the booming trend in employing IoT in cities.

  To address this challenge, the idea of green IoT has recently gained momentum. Green IoT is described as adopting energy-efficient measures to reduce energy consumption and GHG emissions caused by IoT systems in the built environment [175, 176]. In this regard, studies suggested the use of green ICT technologies for green IoT, including the use of green RFID, green wireless sensor network, green cloud computing technology, green M2M, green data centre technology, green communication and networking, and green internet [174,175,176]. The use of drones to help with data transmission is another promising technology for improving energy efficiency in IoT systems. In principle, devices operating within IoT schemes consume high transmission power to transmit data over long distances. Drones can assist with this process by moving close to IoT devices, gathering data, analysing and processing the collected data, and sending it to those devices which are out of the coverage area [176].

• Data analysis: It is evident that IoT analytics provides many benefits, however, these analytics share difficulties during the implementation, specifically in the form of technical challenges. Tibco [177] highlighted two types of challenges including features related to ascertaining time series with data structures and balancing speed and storage. This issue can affect diagnostic and predictive efforts. Alternatively, balancing speed with storage and scaling the process up, especially in the case of time-sensitive data is considered a challenge when historical data is necessary to make comparisons. Studies also underlined issues related to detecting anomalies with IoT data. These anomalies are considered serious complications in the upstream chain and the data ingestion process. As such, there is a need to manage a large amount of data by delivering timely and accurate feedback. Bellini et al. [178] suggested that a promising solution for anomaly detection is to examine IoT data through structure, movement, producer, stack faults, noise, outlier, conditional, typical trends, period, rate, and scalability. On the other hand, there are challenges associated with remote data processing that create issues in centralised computing systems due to high response time and connection loss [50]. For instance, Atitallah et al. [94] showed that false data injection can mislead the analytics processes. Such issues can subsequently lead to incorrect outcomes, guidance, and forecasts [179].

• Privacy and security: The issues related to privacy and security are among the most daunting challenges for implementing IoT in smart cities. These issues are manifested in different layers of IoT architecture such as device level or communication level. At the device level of privacy, there is an issue of “inadequate authorisation and authentication”, “insecure software”, “firmware”, “web interface” and poor “transport layer encryption” [180]. To address this, security considerations should be improvised at different layers of IoT architecture to preclude security threats and attacks [181]. A number of protocols have been developed and implemented on different communication layers to ensure increasing security and privacy in IoT-based systems such as Secure Socket Layer (SSL) and Datagram Transport Layer Security (DTLS) [180]. However, the IoT communication layer is still open to threats imposed by malicious actions mainly due to employing wireless technologies within IoT systems. Therefore, there is an urgency to deploy methods for the detection of malicious activities and activating self-healing measures when threats are identified. Another issue associated with implementing IoT in the context of privacy is that users would feel secure once utilising services provided by IoT systems. Hence, recommendations point out the necessity of maintaining authorisation and authentication via secure networks which enable establishing safe communications between trusted parties [180].

• Interoperability: Interoperability is widely regarded as a challenge in implementing IoT in smart cities. This term refers to the possibility for IoT devices and systems to readily communicate and exchange information with each other without using any particular middleware applications. The root of this issue stems from the heterogeneous nature of IoT systems in which various types of devices and technologies are often deployed for data collection purposes. The issue of interoperability may arise at four levels, including technical, semantic, syntactic, and organizational, as stated by van der Veer and Wiles [182]. Koo and Kim [183] presented five interoperability types, i.e., network, semantic, middleware, syntactic, and security, for which common security should be ensured for each other. Studies proposed solutions to facilitate interoperability in IoT systems such as “adapters/gateways-based solutions”, “virtual networks/ overlay-based solutions”, or “networking technologies” [184]. A study also proposed a hybrid solution, such as ‘Double Obfuscation Approach’, which is comprehensive and reliable in implementing IESCs [185]. Despite these efforts, interoperability still remains a challenge for the implementation of IoT in smart cities.

• Ethical issues: Ethics in IoT applications in smart cities include issues related to social behaviour standards, encapsulating a wide range of challenges such as intellectual property rights, data accessibility, data sharing, and the use of data or information [186]. In a study, Allhoff and Henschke [187] discussed five fundamental issues associated with IoT applications in the context of smart cities, including informed consent, privacy, information security, physical safety, and trust. The study emphasised that these issues intersect in many ways, hence their impacts should be observed in connection with each other. While the ethical requirements vary between countries, Chang [188] proposed an ethical framework that can be used in six smart cities and explained how the framework can be used even in those countries with lower ethical requirements.

• Scalability, adaptability, and reliability: IoT systems provide a large number of services and applications by connecting numerous devices. However, it becomes challenging to design a system that can constantly adapt to the changing needs of users. Scalability refers to the characteristics of a system for accepting the addition of new services, devices, and equipment to its configuration without suffering any interruption or degradation in performance [180, 189]. The scalability characteristic can be vital in helping a system to be competitive, efficient, and capable of delivering sufficient quality of service. In this regard, one of the main challenges for the future development of IoT systems is to become scalable so that such systems can support the integration of a large number of devices with each other having different memory, processing, storage power, and bandwidth [180]. While Artificial Intelligence of Things (AIoT) Initiatives are being adopted to implement smart cities, they are usually ineffective due to a lack of preparedness, resources and capabilities [190]. This study proposed three emerging themes, i.e. proof-of-value, treating and managing data as a key asset and comprehensive commitments, that should be taken into consideration in cities to reduce the challenges of scalability issues.

• IoT and future development of cities: There is a necessity to develop a vision at the government and policy level for incorporating IoT infrastructures when planning for the future development of cities. IoT is fundamental to the realisation of fully functioning smart cities, thus it is important that future urban development plans would encompass smart features (e.g., smart grids, connected homes, telematics, etc.) as a measure to facilitate IoT implementations. Javed et al. [73] emphasised examining smart cities as an integrated network of interconnected systems rather than isolated entities. Since IoT technologies are emerging and many concepts such as climate resilience, net-zero city, climate-intelligent cities, and digital circular economy are continuously influencing the future development of cities at policy levels, the lack of IoT integration within these concepts could have a social and economic impact, including unintended consequences following the adoption of efficiency-improving measures [191, 192].

The literature review of the above seven challenges also indicated the interconnectivity between them which have direct and/or indirect influence on each other. The recommendations that emerged from the literature reviews are also potential solutions for more than one challenge and have a rebound effect. Figure 9 illustrates the interconnected nature of the challenges and solutions in implementing IESC. Figure 9 reveals that recommended solutions may help to mitigate more than one interconnected challenge, however, most challenges also depend on how IESCs would fit and integrate with future technologies.

Fig. 9

Interconnectivity between challenges and potential solutions in implementing IESCs

By summarising the seven key challenges attained using hybrid systematic analysis, several aspects are identified in line with SDGs in terms of Industry Innovation and Infrastructure (SDG9), Sustainable Cities and Communities (SDG11) and Responsible Consumption and Production (SDG12) that can be listed into specific points. (1) The complexity of IESCs reveals limitations with performance across management, implementation and operation at different levels and scales, offering new opportunities to explore this field in-depth. (2) The projected deployment of 50 billion IoT devices around the world by 2030 [20] and the continuous growth of smart cities [21] would impose a devastating environmental impact in terms of e-waste. Hence, efficient implementations regarding circular economy are needed within the boundary of IESCs developments. (3) Data transmission and data storage from IoT devices require considerable electrical power that intensifies energy loads in smart cities. Several solutions were proposed to overcome this issue, but further research is needed to limit this demand to achieve net zero by 2050. (4) Handling and analysing IoT data in smart cities encounter challenges in terms of speed, connection, process, anomalies and forecast that pose a need to improve the reliability of computing in selected IESCs infrastructure to ensure delivering smooth and efficient actions. (5) Data privacy and security in wireless environments are still developing. Protections from any external threats or breaches require further research and exploration to maintain secure authorization and authentication in IESCs. (6) Unclarity or immaturity of standards and guides towards intellectual property rights, data accessibility, data sharing, and the use of data or information are the key challenges in terms of ethical use. Further investigations are needed to enhance transparency in developing frameworks. (7) Exchanging data in different devices and environments without using proper intermediate layers could lead to issues in terms of interoperability as data moves through different levels. Thus, further studies should examine these aspects in detail. (8) Integrating IoT solutions in smart cities features issues in terms of scalability and adaptability to meet users’ needs that are always changing based on the context. Thus, advanced studies to explore the reliability of modifications are required. (9) Several IESCs concepts have been proposed to achieve functionality, however, environmental aspects in terms of future climate resilience were found to be neglected. As a result, accelerating research in this direction is a future demand. (10) Exploring and developing IESCs requires comprehensible vision and policies as stated in several guides to achieve optimum implementation, such as Smart Readiness Indicator and IoT Readiness Level Index, however, further explorations should be extended in providing proper regulations for different industries and experts in this field, especially in the built environment [5].

5 Conclusions

This study has adopted a hybrid literature review technique to identify and critically analyse hot research topics in the field of IESCs. To this end, 843 documents were retrieved from the WoS database and analysed with reflection on the defined objectives. The results of keywords’ co-occurrence analysis in combination with text-mining analysis identified four main areas of IESCs research, including (i) data analysis, (ii) network and communication management and technologies, (iii) security and privacy management, and (iv) data collection.

• From bibliometric analysis and text-mining analysis, the publication trend has shown a consistent upward trajectory in recent years, with 53% of all materials (i.e., 445) published between 2020 and 2022. Considering this steady rise in publications across these periods, alongside the escalating interest in ICT-based technologies, the interest in IESCs is expected to continue growing in the foreseeable future. The findings also revealed that a total of 127 journals collectively published 843 articles spanning the years 2010 to 2022. The top ten prominent journals that have significantly contributed to the advancement of the field, accounting for nearly 51% of the published materials.

• The study examined actionable platforms to enhance the data stream, data management, control data anomaly and improve data analytics. The content analysis of these research areas showed that most data collected via IoT devices is unstructured. Thus, data analytics techniques are required for deployment to process unstructured data for IESCs. Data in IESCs undergoes different complexities that are undertaken based on specific values. The assessment revealed several limitations with IoT data speed, size, accuracy, response and security. It was found that studies are still investigating numerous methods based on the level of processing. The review found that many studies are still exploring and experimenting with different algorithmic techniques for new applications to obtain effective solutions. Finally, achieving integration between IoT and big data analytics shows a promising future but requires further research and investigation, especially in designing smart cities.

• The study found that IoT network management shows no available detailed or comprehensive overview of existing resource-constrained network solutions. The study found that IoT network management needs to incorporate efficient management processes for handling a large number of devices, vast amounts of data, and diverse services with varying requirements. In addition, there are some limitations in terms of developing an effective solution for managing IoT networks that can be challenging due to the inherent constraints of IoT networks. Finally, further studies are required to provide efficient solutions to manage IESCs with low-power networks to handle heterogeneity while ensuring security and privacy and allowing for scalable resource utilisation.

• The assessment of security and privacy management in IESCs revealed issues at different levels of smart cities’ architecture due to the nature of devices being deployed in cities, which are often resource-constrained, thus making cities vulnerable to security attacks. The study identified new technologies that offer high levels of security and privacy, however, they require significant computational resources, which can be a challenge in IESCs with resource-constrained. The study found that areas of predictive computation, security in fog computing, security in smart grid networks and access control in edge computing have specific challenges in addressing particular security and privacy in emerging technologies and infrastructures. Finally, the survey found the need for context-specific solutions to balance the trade-offs between security, privacy, efficiency, and scalability in resource-constrained smart environments in IESCs.

• Data collection in IESCs demonstrated challenges that may vary depending on the smart data applications, data scale and the sector subject to data collection. The study identified 9 areas for data collection, including Energy Conservation, Urban Environment, Health & Wellbeing, Biodiversity, Surveillance/Security & Safety, Transportation & Mobility, Infrastructure & Communication, Tourism and Waste Management. Also, it was observed that there is an extensive range of data collection methods applied in IESCs. Despite the collaborations between IoT sensors and other types of data sources, there is still a lack of consistency when the human factor is involved due to the unpredictability of data and variables.

• The review singled out seven main challenges associated with the implementation of IoT in smart cities for future research. These include energy consumption and environmental issues, data analysis, privacy and security, interoperability, ethical issues, scalability, adaptability and reliability and incorporation of IoT systems into future development plans of cities. The review also revealed some recommendations for those interconnected challenges in implementing IESCs, where most of those issues rely on future/emerging technology and effective integrations within policies to support environmental agendas such as circular economy, climate resilience and net-zero futures.