Computational intelligence techniques for HVAC systems: A review

Buildings are responsible for 40% of global energy use and contribute towards 30% of the total CO2 emissions. The drive to reduce energy use and associated greenhouse gas emissions from buildings has acted as a catalyst in the development of advanced computational methods for energy efficient design, management and control of buildings and systems. Heating, ventilation and air-conditioning (HVAC) systems are the major source of energy consumption in buildings and ideal candidates for substantial reductions in energy demand. Significant advances have been made in the past decades on the application of computational intelligence (CI) techniques for HVAC design, control, management, optimization, and fault detection and diagnosis. This article presents a comprehensive and critical review on the theory and applications of CI techniques for prediction, optimization, control and diagnosis of HVAC systems. The analysis of trends reveals that the minimisation of energy consumption was the key optimization objective in the reviewed research, closely followed by the optimization of thermal comfort, indoor air quality and occupant preferences. Hardcoded Matlab program was the most widely used simulation tool, followed by TRNSYS, EnergyPlus, DOE-2, HVACSim+ and ESP-r. Metaheuristic algorithms were the preferred CI method for solving HVAC related problems and in particular genetic algorithms were applied in most of the studies. Despite the low number of studies focussing on multi-agent systems (MAS), as compared to the other CI techniques, interest in the technique is increasing due to their ability of dividing and conquering an HVAC optimization problem with enhanced overall performance. The paper also identifies prospective future advancements and research directions.


Energy consumption in buildings
Globally, buildings consume approximately 40% of the total energy used and contribute towards 30% of the total CO2 emissions (Costa et al. 2013;Shaikh et al. 2014). The building sector is considered the largest consumer of energy in the European Union (EU) and is responsible for up to 40% of the total energy consumption and 36% of greenhouse gas (GHG) emissions (Grözinger et al. 2014). In the UK and USA, buildings consume approximately 39% (Pérez-Lombard et al. 2008) and 41.7% (EIA 2011) of the total energy used respectively. Majority of this energy come from the burning of fossil fuel, which amounted to 81.23% of global energy consumption in 2011 (The World Bank 2014). Associated GHG emissions from the burning of fossil fuels have been attributed as the extremely likely cause of anthropogenic climate change (IPCC 2013). The building sector, therefore, plays a significant role in mitigating the impacts of climate change by reducing GHG emissions from burning fossil fuel for energy. The European Commission recognizes that the improvement of the energy performance of Europe's building stock is crucial for meeting both short-(20% by 2020) and long-term (88%-91% by 2050) targets of significant GHG reductions from 1990 levels and the move towards a low carbon economy by 2050 (EC 2011). The evolution of EPBD, from its approval in 2002 to the recast in 2010, illustrates a marked move towards more stringent requirements for building energy efficiency, in particular for building energy systems such as heating, ventilation and air-conditioning (HVAC). In a recent report on the European building stock (EU27, Switzerland and Norway), the Building Performance Institute Europe (BPIE) suggested that a greater emphasis should be placed on the implementation of smart energy management to drastically reduce energy use, especially in non-domestic buildings (BPIE 2011). The report also estimates that the average specific energy consumption in the non-domestic sector in the surveyed countries is 280 kWh/m 2 per year, which is at least 40% higher than the equivalent value for the residential sector. These figures highlight the importance of HVAC in energy demand and the need for further research and development in HVAC energy efficiency to achieve societal goals of low energy use and GHG emissions.

HVAC systems, optimization, and fault detection and diagnosis
In most HVAC operation scenarios energy conservation is desired while maintaining the occupant thermal comfort level within a predicted mean vote (PMV) range of ±0.5 (ASHRAE 2009;CIBSE 2006), which corresponds to 10% predicted percent dissatisfied (PPD) (ISO 2005). A combination of these two desired outcomes is typically used as objectives to accomplish operation tasks. Various system parameters that have an impact on the desired objectives are used as design variables, the values of which are changed to maximise or minimise the objectives. For HVAC system, there are two types of controls: local and supervisory controls. In local control type, the design variables that are considered in the optimization or control problem formulation can include valve position, damper position etc. Whereas, on supervisory level the control variable can be temperature set-points and schedules. Figure 1 illustrates a typical air handling unit of an HVAC, comprising: an outdoor air damper to control outside air intake; an exhaust air damper to control discharge to outside; a return air damper to control recirculation; a return air fan to control extraction of indoor air; heating and cooling coils to increase/decrease air temperature; and a supply air fan to control flow of conditioned air. Further components such as filters, dehumidifiers/humidifiers, reheaters, etc. can be added depending on circumstances but are excluded in this article for brevity. System variables are discussed here with the help of four scenarios, illustrated in Fig. 1. Scenario A illustrates the control of supply and/or return airflow rate to maintain desired comfort conditions. Air supply flow rate can also be constrained to avoid localized discomfort due to draught. Scenario B aims to reduce energy consumption while maintaining comfort conditions by controlling the supply air temperature through heating and/or cooling coils. The operation of heating and cooling coils is typically linked with room air temperature, which together with room air humidity affects thermal comfort level. Changing the room set-point temperature affects energy consumption and thermal comfort. Widening the temperature range of thermal comfort by approximately 1 °C can have a corresponding effect on thermal comfort by ±0.5 PMV (CIBSE 2006). Scenario C is concerned with minimising thermal energy consumption by controlling the return air damper, while maintaining a desired level of thermal comfort. Scenario D involves the control of outdoor and exhaust air dampers to control how much fresh air is introduced, which has a corresponding effect on thermal energy consumption. The control of dampers can also be constrained to meet the minimum fresh air requirement to dilute air pollutants and odour for acceptable indoor environments. Indoor air quality (IAQ) in the form of carbon dioxide (CO 2 ) concentration can be considered as an optimization objective. Scenarios C and D are often combined together to control the mixing of return air with outdoor air.
Fault detection and diagnosis (FDD) of HVAC systems plays an important role in improving energy efficiency, thermal comfort and reducing maintenance and operating costs. Its basic aim is to detect outlier that may represent fault in an HVAC system. According to Du et al. (2014b), in general, FDD methods can be divided into three categories: the rules-based, model-based and data-driven methods. Model-based can be developed by employing energy and mass balance phenomenon and residues can be calculated by comparing outputs from the model and actual measurements. Rules-based methods do not need any model of a system and rely on the expert knowledge to create expert rules. HVAC systems/its components are controlled by different controllers, which can be stand-alone controllers or connected to a sophisticated building automation system (BAS). BAS consists of many sensors and controllers, and a large amount of data is available on the BAS central station. This rich data gives an opportunity to use it for developing datadriven fault detection and diagnosis strategies to distinguish between faulty and non-faulty operating conditions. The data-driven methods do not require any physical model or expert knowledge of the system. In Fig. 1, a fault may occur in outdoor air (OA) damper. For example, for certain conditions (e.g. weather and indoor), the outcome from the FDD model/strategy is always 30% open, but the current measurement of OA damper position is 90% open. This information can result in a warning suggesting that there may be outdoor air temperature sensor and/or economizer control fault(s).

Evaluation of objective functions
Indicators for building performance (e.g., energy use) typically have non-linear time dependent relationships with control variables in HVAC systems. It is challenging to define and use a straightforward mathematical relationship between inputs and outputs due to the complex behavior of the system and dynamic nature of the problem. Different types of prediction engines are used with computational intelligence (CI) techniques to evaluate optimization objectives. Prediction engines can be broadly classified into three:  Analytical;  Numerical (e.g., whole building simulation); and  Predictive (e.g., ANN).
Whole building simulation engines such as EnergyPlus 1 , TRNSYS 2 , DOE-2 3 , etc. often enable the evaluation with reduced uncertainties, primarily due to their multi-domain (thermal, lighting, network airflow, etc.) modelling capabilities (Mourshed et al. 2003), as well as finer spatial and temporal resolutions of these tools. However, the disadvantage of using simulation tools as evaluation engines is the computation time required for simulation, making them unsuitable for online or near real-time applications. To reduce computation time, predictive models such as artificial neural networks (ANN) are often used, in particular for online control optimizations. However, it is difficult to find the right topology for an ANN, which requires several experiments to determine the best combination of an ANN based predictor. The topology may also vary from problem to problem, making it necessary to develop case-by-case ANN predictors, with added complexities for scaling up.

CI algorithms
Computational intelligence (CI) was first proposed by Bezdek and the term was first used by the Institute of Electrical and Electronics Engineers (IEEE) Neural Networks Council in 1990 (Bezdek 1998). There is no commonly accepted definition of computational intelligence in the literature (Siddique and Adeli 2013). Siddique and Adeli (2013) defined computational intelligent system as a system which deals with low-level data such as numerical data, has a patternrecognition component and does not use knowledge in the artificial intelligence (AI) sense, and additionally when it begins to exhibit computationally adaptivity, fault tolerance, speed approaching human-like turnaround and error rates that approximate human performance. CI is a rapidly advancing research field and includes a collection of various computation techniques, including but not limited to: expert systems, genetic algorithm (GA), artificial neural network (ANN), support vector machines (SVM). The most commonly used CI techniques for HVAC applications are fuzzy logic (Chu et al. 2005;So et al. 1997;Zheng and Xu 2004), ANN (Argiriou et al. 2000;Curtiss et al. 1994;Kanarachos and Geramanis 1998), GA (Lu et al. 2005;Mossolly et al. 2009;Nassif et al. 2005;Wang and Jin 2000;Wright et al. 2002), multi-agent systems (Hagras et al. 2008;Rutishauser et al. 2005;Yang and Wang 2013) and pattern recognition-based methods (Du et al. 2007a;Hu et al. 2012;Naja. et al. 2012; Wang and Cui 2005;Wang and Xiao 2004a;Zhao et al. 2013b). CI techniques have been successfully applied by researchers to overcome time delay, system uncertainties, and nonlinear feature in PMV calculations (Dounis and Caraiscos 2009), as well as includes HVAC applications such as prediction, optimization, control and fault detection and diagnosis. According to their applications, CI techniques can be classified into several groups, as shown in Fig. 2. Artificial neural network and support vector machine have been utilised for prediction, control and classification purposes. In literature, most of the applications for the HVAC systems are mainly based on stochastic based optimization algorithms e.g. evolutionary algorithms and their enhancements, as presented in this paper. For control and fault detection and diagnosis purposes, fuzzy logic based controller and detectors are widely used in the literature. Pattern recognition-based methods are mainly applied for fault detection and diagnosis purposes. Multi-agent systems can be used for many purposes such as control, monitoring and detection. For HVAC systems, multi-agent systems were mainly used for control purposes.

Study contents
Different literature surveys on optimization techniques for energy applications in buildings have been published. Dounis and Caraiscos (2009) reported on control systems for energy management and thermal comfort in buildings with a brief discussion on multi-agent systems. In a recent publication, Evins (2013) reviewed optimization methods for sustainable building design as a broad topic, including envelope design, configuration and building control. The Fig. 2 Classification of computational intelligence techniques with respect to their primary area of application. Hybrid algorithmscombinations of two or more algorithms (e.g., ANN-GA) are not explicitly illustrated here to avoid compromising the clarity of the image. Algorithms in shaded boxes are reviewed in this article coverage in terms of the breadth of application areas constrained a deeper exploration of HVAC systems and algorithms. Another review on optimization methods applied to renewable and sustainable energy was published by Baños et al. (2011). Katipamula and Brambley (2005) reviewed different methods applied for fault detection and diagnosis in HVAC systems. A review of fuzzy logic systems applied to building research was presented by Kolokotsa (2007), where the author presented fuzzy logic applications for indoor visual and thermal comfort, and indoor air quality, as well as fuzzy logic control systems. Various methods for control were also discussed by Shaikh et al. (2014). The existing body of published reviews lack a comprehensive and focussed discussion on HVAC optimization, one of the most promising application areas for both classical optimization and CI techniques. Moreover, the choice of evaluation engine is critical for HVAC applications during the operation stage due to the time-critical nature of the application. The literature also lacks in a critical discussion of optimization variables and objective functions adopted in various research.
Optimized control/operation of HVAC systems is challenging due to the presence of system non-linearities and delays, as well as seasonal variations in weather conditions. HVAC problems are classed as non-polynomial hard (NPhard) problems. CI techniques are found to be effective in dealing with NP-hard optimization problems with incomplete information, as opposed to classical optimization techniques such as gradient-based methods. This research is, therefore, aimed at filling the gap in literature through a comprehensive and critical review of the theory and applications of CI techniques for prediction, optimization, control and diagnosis of HVAC systems. Our work focuses on the widely used CI algorithms: artificial neural network (ANN), genetic algorithm (GA), evolutionary programming (EP), ant colony optimization (ACO), particle swarm optimization (PSO), pattern recognition-based methods (principal component, clustering, pattern matching and Bayesian networks) fuzzy logic and multi-agent systems (MAS).The paper does not cover HVAC system sourced by renewable energy systems.
The paper started with a brief discussion on the rationale for the use of computational intelligence in HVAC applications, followed by a discussion on the evaluation of objective functions along with classifications of CI algorithms based on their application in the HVAC domain. The rest of the paper is structured as follows. The methodology adopted for this review is discussed next. The review itself is organized in six sections, five of which are dedicated to key CI algorithms: metaheuristic, artificial neural networks, pattern recognition-based methods, multi-agent systems and fuzzy logic. The sixth section is dedicated to hybrid algorithms that combine one or more of the CI techniques discussed. The review sections are followed by an analysis of trends in published literature. Conclusions are drawn in the end and future directions for research are discussed. A discussion on the formulation of an optimization problem using illustrated optimization scenarios is also discussed in the appendix. We feel that we have presented a comprehensive review on CI techniques applied to HVAC systems and have made every effort to include all research studies in this domain but as no review can be exhaustive and there will always be some studies that fail to be included.

Methodology
The review was conducted by the authors over a period of fourteen months and relied on: (a) searching publication databases for peer-reviewed journal and conference articles, and books, and (b) researchers combined experience in the development and application of building optimization spanning several decades. The literature search was carried out using relevant keywords, identified through an iterative process of exploration, brainstorming and selection. Three different categories of keywords were used: (a) CI technique (e.g., neural networks), (b) application (e.g., fault detection), and (c) generic terms (e.g., building optimization) to ensure the breadth and depth of coverage. Search terms were combined with Boolean operators, "OR" and "AND" to cover widest possible combinations in search engines. Five key search engines were used to identify potential sources: IEEE Xplore (http://ieeexplore.ieee.org/Xplore/home.jsp), ScienceDirect (http://www.sciencedirect.com), Scopus (http://www.scopus.com/home) and Google Scholar (http://scholar.google.co.uk). Cited articles were checked and relevant cited articles were included in the review.

Metaheuristic algorithms
Metaheuristic algorithms are able to find local minima/ maxima; however, they can not guarantee an optimum solution for non-polynomial hard (NP-hard) problems. In literature, metaheuristic algorithms are typically classified based on their search techniques, i.e., as population-based or single individual-based (Yuce 2012). The most popular population-based algorithms are genetic algorithm, evolutionary programming, ant colony optimization, particle swarm optimization, and the bees algorithm. The most popular single individual-based metaheuristic algorithms are tabu search, simulated annealing and stochastic hill climbing. Four metaheuristic algorithms: genetic algorithm, evolutionary programming, ant colony optimization and particle swarm optimization are reviewed in this section because of their wider use in HVAC applications.

Genetic algorithm
Genetic algorithms (GAs) are an adaptive heuristic search technique based on the process of natural selection (Bagley 1967;Holland 1992). GA gives a set of optimal/potential solution(s) to a problem. Each solution in the population is known as an individual. A generation is a new population of individuals that is created each time the optimization algorithm is repeated. GA has three evolution operators: reproduction, crossover and mutation. These operators control the evolution of future generations. Crossover involves swapping of two randomly chosen chromosomes to create a new individual or offspring. The mutation operator is also inspired from nature to generate modifications on an allele in order to look for new points in solution space (Goldberg 1989). GA is often initiated with a random population but can also be designed to start from a known population, i.e., a known set of individuals. The algorithm evaluates the population and then the three GA operators are applied to generate a new set of population. To evaluate a single individual, GA utilises a cost/fitness function that measures the performance (i.e., fitness) of the solution. The probability of an individual to be selected for next generation depends on its fitness value and the selection process. Fitness proportionate selection, also known as roulette wheel selection, is a commonly used selection technique that involves the following steps (Sahu et al. 2012): (1) Normalization of each individual's fitness value; (2) Sorting of the population by descending fitness values; (3) Computation of the accumulated normalized fitness values; (4) Generation of a random number between 0 and 1; and (5) Selection of the first individual whose accumulated normalized value is greater than the generated random number.
The individual with the higher fitness value has a higher probability than the lower one. The overall process of a typical genetic algorithm optimization is illustrated in Fig. 3. There are many variants of GA depending on the implemented selection and reproduction methods, as well as optimization strategies such as elitism 4 . Further discussion on GA fundamentals is out of the scope of this article; hence, the reader is referred to (Goldberg 1989;Mitchell 1996). Zhou et al. (2003) incorporated an optimization module into EnergyPlus to determine the best control strategies to reduce electricity cost by varying cooling set-points for different seasons. The authors compared different optimization methods: Nelder-Mead simplex method, quasi-Newton, simulated annealing (SA) and GA. Although SA gave better performance during summer, GA performed better during the remainder. Computation time for SA in the study was about 7 times higher than that for GA, which had the second largest computation time. GA uses a population-based global search methodology, whereas the other algorithms in this study perform a single individual-based strategy to look for the optimum that may be cost effective for a simple problem with non-complex solution space. However, it may be useful to implement a population-based solution for complex problems such as energy optimization of HVAC systems.

Single-objective GA applications
A genetic algorithm was used by Lu et al. (2005) to optimize the overall system energy consumption of an HVAC system. Adaptive neuro-fuzzy inference system (ANFIS) was used to model the duct and pipe networks and to obtain optimal differential pressure set-points. The authors selected simple but accurate component models for real-time system optimization. The results obtained from  optimized approach were compared with tradition control schemes (fixed chilled water supply temperature control and fixed differential pressure set-point control). It was found that the overall performance of the system was improved by using the proposed method. The authors did not include any statistical analysis of the proposed method to demonstrate its robustness. The running cost of an HVAC system was minimised by using online optimal control strategy with GA (Wang and Jin 2000). Dynamic models were developed to self-tune the systems and a GA was used to tune the model parameters (control parameters). This method allowed the user to select different weight factors in the objective function and thus giving more flexibility to the occupants. The authors compared the proposed method with a conventional method. According to the experiments, the proposed method performed better than the conventional method. This is due to utilising an optimization approach onto the controller. However, the robustness of system has not been highlighted. The experiments can be extended by including several parameter combinations to verify the robustness of the proposed methodology.
A system-based approach to optimize total chiller energy cost by using GA along with ANN was proposed by Chow et al. (2002). The authors initially considered chilled and cooling water mass flow rate, chilled water temperature, cooling water return temperature as control variables of the problem. The proposed approach considered three different strategies by using each control variable. In the first strategy, both chiller and cooling water flow rates were kept constant. In the second strategy, they used variable cooling water flow rate and constant chilled water flow rate; and in the last strategy both these variables were varied. ANN was used as a fitness function predictor and 5-5-9-4 architecture was selected for the ANN. The proposed cost function was an aggregated calculation of the outputs (Chow et al. 2002). It was found that the highest energy saving with this optimization process was achieved with the third strategy i.e. by using a maximum number of control variables. It is worth mentioning that the accuracy of the optimization process is also strongly related to the cost function predictor. Therefore, the authors should have made efforts to find the best topology for the cost function predictor to obtain better results. In the proposed model, the MSE (mean square error) of the ANN was around 0.002, which may reduce after network's tuning.
Another, chiller optimization model using GA was proposed by Čongradac and Kulić (2012). They also utilised ANN to predict the outputs. In the proposed ANN, they utilised outlet temperature, return temperature and external temperature in the chiller to predict the status of four compressors used in the study. GA was used to find optimal input variables for the ANN for minimising energy consumption. The proposed model was saving 12% more energy compared to the normal daily usage of the model. To verify the performance of the algorithm the results were also compared with EnergyPlus simulation model, and it was found that the proposed model performed better than the simulation model. However, the authors did not present information about computational time and statistical analysis of the optimizer. Moreover, the experiments can be extended by using different configurations of the GA. Counsell et al. (2013) designed a robust non-linear controller. To improve its robustness, a non-linear inverse dynamics (NID) technique was combined with a GA. The controller results were compared with a GA based PI controller, and it was found that the proposed controller was more stable, faster in response and had no oscillatory behaviour. Ma and Wang (2011) proposed an optimized control using simplified linear self-tuning strategy with GA for a central chiller plant to minimise its energy consumption. The proposed model was tested on a simulated virtual test system by using TRNSYS simulation environment. In the proposed model, temperature controllers were utilised to optimize the overall system. The results for mild-summer and sunny-summer conditions of the optimized controller were compared with the conventional controllers and found that the highest energy saving achieved during spring season was about 2.5% lesser than the conventional control strategy. The lowest energy saving achieved during the sunnysummer seasons was about 0.7%. The author did not tune the parameters of the optimizer which resulted in lower energy savings during the sunny-summer conditions. Also, the more energy savings can be achieved by addressing the model mismatch issue.
Evolutionary algorithms can also be used for HVAC system designing (Stanescu et al. 2012). A detailed simulation tool (DOE-2) was used to evaluate the objective function and the authors also used three different permutation options (only mutation, mutation + crossover and only crossover). The optimizer was aimed to find an optimum HVAC system configuration to reduce energy consumption. It was found that the strategy of using crossover and mutation saved more energy as compared to the other strategies. However, the authors selected mutation only strategy as their optimal solution because it resulted in a good compromise between computational time and optimization results. This strategy consumed 1.76% more energy, whereas the strategy with crossover and mutation took 1.5 more days to complete 500 iterations. A better solution can be achieved with better parameter sets of the EA, as there are no unique parameter sets for EAs to find better solutions. Therefore, parameter optimization is also required for the optimizer itself to obtain a better solution.

Multi-objective GA applications
A multi-objective genetic algorithm (MOGA) method to find an optimum trade-off between energy cost and thermal discomfort was proposed by Wright et al. (2002). A single zone and an HVAC system, which consisted of heating and cooling coils, fan and heat exchanger, were simulated for three days. There were 189 control variables in the problem formulation, and it was pointed out that reducing the number of control variables may be less effective for lightweight buildings. Different constraints were imposed on coil design, system capacity and fan performance limitations. The system was simulated for three days to take into account the swings in the weather data. The results of each individual were compared to the results of single objective based optimization solution. As expected, the results of single objective optimizer were better than the multi-objective optimizer. Moreover, the study can be extended to find a better solution with different parameters settings. As GA is a global search algorithm and cannot guarantee a better solution with any random parameter sets for such a complex problem. Nassif et al. (2005) proposed an HVAC optimization method to optimize both energy consumption and thermal comfort by using a non-dominated sorting genetic algorithm (NSGA-II). Supply air and chilled water supply temperatures, minimum outdoor ventilation, supply duct static pressure, reheat (or zone supply air temperature) and zone air temperatures were optimized in the proposed model. The results from actual and optimal energy use were compared and it was found that the optimization strategy could save 16% of energy for two summer months. The results of the proposed algorithm were also compared with the results of a single objective (electricity consumption) with a constraint on thermal comfort, and it was found that the proposed algorithm performed better. This is an expected result; however this can be related to the number experiments considered in this study. Mossolly et al. (2009) proposed a GA based optimization process for a VAV air conditioning system to optimize energy cost and thermal discomfort was implemented by using a quadratic mathematical cost function. Three control strategies (one conventional base strategy and two optimized advanced strategies) were employed and simulated to ensure thermal comfort with less energy cost. Simulation results showed that 30.4% of the energy was saved by using the optimized control strategies during the summer season. This was because of using fixed set-points in the conventional system; however optimization based process becomes more proactive to changes in the VAV system. Also, studies with fixed variables were also included but they were not justified as the best combination of the parameter sets for the proposed problem. Therefore, by using a better combination of the parameters' set may achieve better results.

Evolutionary programming
Evolutionary programming (EP) shares same algorithmic principles with GA by applying a similar strategy to converge to optimal solutions. The main difference between them is their data representation, selection approach and the importance of recombination and mutation (Fong et al. 2006). GAs use binary representation, whereas EP uses real numbers for the problem variables. In EP, both mutation and crossover are the core operators, while in GAs crossover is used as a core operator and mutation is used as a background operator. GA uses a strategy called "selection for reproduction"; on the other hand, EP's next generation selection is based on "selection for survival" (Fong et al. 2009). The EP procedure is shown in Fig. 4 Generation of initial population tree structure   Fong et al. (2006) proposed an evolutionary programming (EP) based optimization process to reduce the energy consumption of an HVAC system for a local subway station by using TRNSYS simulation tool as evaluation engine. The authors studied three different strategies to analyze the effect of different control variables on total energy consumption, including chilled water temperature and supply air temperature of air handling unit (AHU). The first strategy was the minimisation of the year-round energy consumption of the HVAC system using chilled water temperature as a control variable. The second strategy was to enhance the first strategy by including one extra control parameter, i.e., supply air temperature. The last strategy was to minimise the monthly energy consumption using both control variables used in the second strategy. It was found that by using two control variables gave better solution and also, monthly based optimization process generated better results compared to the yearly based optimization. The main effect to have a better solution with two control variables is directly related to optimizing the environmental conditions using two variables. Control variables tend to reduce entropy in the overall system and also more control variables in the system also increase the gain of the overall system. The performance of the monthly based solution was found better than the one with a yearly solution because the resolution was better on monthly based solution. The authors used 50 epochs for the problem but did not mention about the computational time required to solve the problem.

Ant colony
Ant colony optimization (ACO) was first proposed by Marco Dorigo in the early 1990s (Colorni et al. 1991) and can be categorised as a metaheuristics method. It uses an algorithm that is used by ants in real life. The basic ant colony concept is shown in Fig. 5, the black dotted lines are the amount of pheromones on each trail. The behaviour of ants to follow optimal path to search for their food can be explained in four steps (Fig. 5): (a) Initially all three ants can choose any of the paths with same probability to reach to their food. (b) All of the ants chooses a different paths and one ant has a shorter path than the others. Ants deposit chemical substance (pheromone) while they walk so that other ants can follow them and also to remember their path. (c) The shorter path has a stronger pheromone trail than the longer path. (d) After certain time, the longer path pheromones were evaporated, and the shorter path pheromone trail became more dominant and all ants will The ACO algorithm is demonstrated in Fig. 6, the ACO procedure consists of mainly three steps. In the first step initialisation of pheromone takes place. The second step involves the creation of complete solution by all ants. The solution is created by using pheromone trail. The quantity of pheromone trail is updated in the third step. This step is applied in two phases: in first phase, a fraction of pheromone evaporates (evaporation phase) and then in reinforcement phase each ant deposits pheromone that is proportional to the value of fitness of its solution. After this step, step 2 is again applied to the problem until the stopping criterion is met.
In ACO, the quantity of pheromone is intensified around the best objective function value that was obtained during the previous iteration. The new position of ants is given by Eq. (1): ,best 1 where: x ¶ is a randomly generated vector in the range of [−α, α] and ,best g t x is the best solution found by ants. The length of jump is calculated by using Eq. (2): The  sign in Eq. (1) is the direction of movement, and it is decided depending on the value of best x comparing to ,best g x . The step 3 of ACO procedure is described by Eqs. (3) and (4): where: τ t is the quantity of pheromone, Eq. (3) represents evaporation phase and Eq. (4) represents reinforcement phase.  Lixing et al. (2010) proposed an ACO based method to predict HVAC cooling load by using support vector regression (SVR), SVR is known for its better ability to correlate inputs and outputs. However, the parameter determination of SVR is a big challenge and ACO was used to deal with this problem. The authors utilised this approach to determine following parameters of the SVR to predict the cooling load: the penalty parameters, insensitive loss function and a kernel function. The results of the SVR-ACO were compared with the back propagation ANN (BPANN). According to the experimental results, the optimized SVR produced better solutions than the BPANN. However, the authors did not compare other training algorithms with BPANN. A Levenberg-Marquardt based BPANN could have been a better alternative to generate better solution because it uses least square algorithm as learning and training methodology.

Particle swarm optimization (PSO)
PSO is categorised as a heuristic search method and is a population-based approach. It was invented by Kennedy andEberhart in 1990s (Kennedy andEberhart 1995). The steps involved in PSO algorithm are illustrated in Fig. 7, and are: generating particle positions and velocities, updating their velocity and updating their position. The particle is a point (solution) in the design space; each particle is initialised by a random position in the problem space and then is "flown" through the space to find the best position for itself. The solution of each particle is compared with other particles by using a fitness function. After this comparison, each particle moves toward two positions i.e. its own best position and the best position achieved so far by all particles (Lee et where: C 1 and C 2 are the acceleration constants, r 1 and r 2 are two random numbers between 0 and 1, v i and x i are the velocity and position of the particle i, respectively, y i is the personal best position of particle i and y g is the best position of all the particles (at present).

PSO Applications
Single-objective applications. Xu et al. (2013) used PSO to pre-cool a building for energy reduction by shifting peak load of the building. The PSO based optimizer utilised the start time and duration of the pre-cooling/pre-heating to minimise the energy consumption. The energy consumption of the case with pre-cooled strategy using PSO optimizer was lesser than the one without PSO optimizer. PSO algorithm is one of the efficient stochastic based optimization algorithms and if the weights and inertia parameters are properly tuned then the computational speed can be faster. Lee and Cheng (2012) combined EnergyPlus simulation programme with a hybrid optimization algorithm, combining PSO with Hooke-Jeeves, to find optimal settings of a chilled water system. The PSO was used for local search optimization problem; however, algorithm had slow convergence rate and weakness on the search process (Yuce et al. 2013). Therefore, PSO was combined with the Hooke-Jeeves algorithm to avoid its weaknesses in this research. Two types of strategies were implemented to show the strength of the algorithm. In the first strategy, constant optimal chilled and cooling water temperatures were selected as set-points. Whereas in the second strategy, both set-points were considered as changing over time. Simulation results for 4 summer and 4 winter days showed that the energy consumed was reduced by 9.4% in summer and 11.1% in winter when compared with the conventional settings. This study was only carried out for 8 days, which limits its usefulness and an yearly based simulation should have been performed to see the effect of seasonal variation.
Multi-objective applications. An optimization of an HVAC system by using strength multi-objective particle swarm optimization (S-MOPSO) was carried out by Kusiak et al. (2011). The proposed algorithm was a combination of multiobjective particle swarm optimization and strength pareto evolutionary algorithm (SPEA). Evolutionary algorithms have strong ability of global search and weak ability of local search. On the other hand, PSO has a strong local search ability and weak global search ability. The hybridization of these both algorithms empowers the search process. Hourly optimal control settings were generated to minimise the energy consumption while maintaining the thermal comfort at an adequate level. The proposed algorithm performed better than the conventional MOPSO, because of its enhanced ability to search for both local and global solutions rather than only local solutions.

Discussion
Several stochastic based optimization algorithms have been utilised in the area of HVAC system optimization. Each algorithm has different specifics and search ability on the solution space; for example GA and EA are suitable for global search, however, the convergence rate is lower than the other algorithms. Moreover, PSO has a strong ability on the local search optimization process and therefore the search process is slower than GA. Further, ACO is capable to do a global search and has a better convergence rate than GA on the local search process. According to Mitchell (1996), the GA method is more suitable when an optimal solution is not perfectly smooth. Also, when the cost function is noisy or optimal problem is not well understood. GAs are mostly applied when a problem does not require an absolute solution. In some studies, GAs were also combined with artificial neural networks. ANNs were used for modelling purposes due to their ability to model complex non-linear system, whereas GAs were used to find a global optimum e.g., in the studies by Chow et al. (2002) and Čongradac and Kulić (2012). More work needs to be done to integrate control strategies and models with the building energy management systems. Real-time experiments need to be performed to evaluate the performance of control strategies. Table 1 summarizes work focussing on meta-heuristics algorithms.

Artificial neural networks
The idea of artificial neural networks (ANNs) is inspired from a human nervous system with its neurons, axons, dendrites and synapses. A neural network is a parallel distributed processor that stores knowledge from experience and makes it available to use (Haykin 1994). The artificial neural network resembles the human brain in two ways: the network acquires the knowledge through the learning process, and inter-neurons connection strengths (synaptic weights) are used to store the knowledge. Artificial neural networks do not require any advanced information about the system as they operate like a black box model. ANNs learn the relationship between inputs and outputs through previously recorded data. In literature, several ANN learning strategies have been introduced such as feed-forward network, self-organizing maps, Hopfield Network, Elman Network and Radial Basis Network (Krenker et al. 2011). The most popular and generic among them is the feed-forward network, which is used for most of the problems, other learning strategies are specifically designed for specific problems such as classification, clustering and mapping (Mokhlessi et al. 2010). A schematic diagram of a feed-forward neural network architecture is shown in Fig. 8. A neural network consists of input, output and hidden layers. In Fig. 8, only two hidden layers are shown and the number can be more than 2 depending on the nature and complexity of the problem. At input layer, each neuron corresponds to each input parameter and at output layer each neuron corresponds to each output parameter (Kalogirou 2009). Each neuron is connected to every other neuron of the previous layer through adaptable synaptic weight. A training process is carried out to train ANN by modifying the connection weights and weights are adjusted to produce the desired outputs. The procedure of artificial neural network is shown in Fig. 9. The output signal from an ANN is calculated by using Eq. (7): where i is used for i-th input, W is the connection weight, T and b represent transfer function and bias value respectively, a denotes input data and y is the output signal.  (2011) Notes: a EP: evolutionary programming, NM: Nelder-Mead simplex, BFGS: Broyden-Fletcher-Golfarb-Shanno, GA: genetic algorithm, SA: simulated annealing, PSO: particle swarm optimization, S-PSO: strength-particle swarm optimization, EA: evolutionary algorithm. b Control and/or fault detection and diagnosis (FDD) method. ANN: artificial neural network, PI: proportional integral, ANFIS: adaptive network-based fuzzy inference system, NID: non-linear inverse dynamics.

Artificial neural network topology
The most important design considerations for an artificial neural network are finding the right number of hidden layers, the number of neurons in hidden layer, and the number of input and output nodes. According to Kalogirou (2009), finding the right number of neurons in the hidden layer is the biggest challenge in designing an artificial neural network. Their selection depends on the number of inputs and outputs and also on the number of training sets/cases. In past, researchers have tried to find the right number of hidden layers and hidden layers' neurons; however it is more a trial and error approach. Too few neurons in hidden layer can result in large errors, whereas too many neurons can result in overtraining. Initially, a lower number of hidden layer is selected and then the ANN is trained and tested. Depending on the results obtained, the number of hidden layers can be increased. The number of neurons also depends on the complexity of the problem. Argiriou et al. (2000) used an ANN network for solar radiation prediction of solar irradiation that consisted of 28-16-8-1 neurons. Solar irradiation is a complicated variable as it changes both in space and time and therefore extra care is needed in selecting network topology. The authors considered last and six previous values of daily normalised time, ambient temperature, day of the year and solar irradiation. The higher number of input neurons could not guarantee better results and can increase the computational time. Moon et al. (2013) compared the performance of an ANN model with different hidden layers (from 1 to 5). It was found that the minimum RMS (root mean square) and MSE (mean square error) values were produced by a 3 hidden layered model. The authors also varied number of neurons in the hidden layers and a variation from 10 to 20 neurons in each hidden layer was tested, and it was found that the optimal number of neurons was 10 neurons. This shows that increasing the number of neurons may not give better results but can increase the complexity of the model. Different topologies were adopted by different researchers for ANN applications on HVAC systems. A summary of these topologies is given in Table 2.

Training methods
ANN learns from examples and generates a mapping relationship between inputs and outputs. To generate this mapping relationship, every network is trained from the given example, and then testing and validation are carried out. The training methods for ANN are mainly classified into two main groups i.e. supervised learning and unsupervised learning. In supervised learning, the inputs and desired outputs are known to the network and training methods are applied to minimise the error between the desired outputs and ANN output values. In unsupervised learning (adaptive learning), inputs are only known in the topology and learning methods are used to find the hidden structure in an unlabeled dataset.
In literature, several ANN training algorithms have been proposed. Yuce et al. (2014) tested several training algorithms to increase the efficiency of a neural network for HVAC control system. They compared the training performance of the gradient descent based algorithms, Levenberg-Marquardt Algorithm (LMA) and conjugate based training algorithms. The gradient descent based algorithm utilises steepest descent algorithm, whereas LMA is an approximation of the Newton method that increases the accuracy of the output compare to other algorithms (Hagan and Menhaj 1994). The  Moon et al. (2013) 2013 Indoor air temperature 6 7-10-10-10-10-1 Wang and Chen (2002) 2002 Stuck damper 3 3-10-1 Outdoor airflow sensor fault Supply and return airflow sensors fault CO2 sensor fault Curtiss et al. (1994) 1994 Energy load 4 6-10-10-1 Ning and Zaheeruddin (2010)  2010 Fan speed 3 7-40-5 Airflow rate entering into zones Chilled water flow rate Compressor speed Morisot and Marchio (1999) 1999 Supply air temperature 3 6-4-2 Supply air humidity Lee et al. (1996Lee et al. ( ) 1996 Supply fan 3 7-5-9 Pump Return air fan Cooling coil valve Thermocouple Pressure transducer Supply and return flow stations Wang and Chen (2002) 2002 Outdoor airflow rate 3 3-10-1 Li et al. (1996) 1996 authors proposed that the second order gradient methods (Levenberg-Marquart) are best suited for complex problems such as HVAC control . A detailed mathematical description of different learning algorithms can be found in Meireles et al. (2003). According to Werbos (1974), the most popular learning algorithms for ANNs are the back propagation (BP) and its variants. The BP learning algorithm is a gradient descent algorithm that reduces the total error by changing the weights along its gradient to improve the performance of an artificial neural network. The sum of square value of the error is calculated by Eq. (8): In the above equation, t is the target output, o is the output vector and E is the sum of squares error function.

HVAC control
Artificial neural networks are self-learning controllers that can be used for energy management and comfort management. An ANN based controller to optimize building energy demand by predicting the energy demand and weather conditions was proposed by Argiriou et al. (2000). The controller was designed for a solar house and was used to decrease the possibility of overheating. The authors used four modules for predicting solar radiation, outside air temperature, electrical heaters' state (ON/OFF) and for estimating indoor air temperature for next time step. The controller was implemented on PASSYS test cell with a time step of 15 minutes (Vandaele and Wouters 1994).
Considering the complexity of the controller, 15 minutes time step was a reasonable selection and as maller time step might have increased the complexity of the hardware. The controller was compared with an ON/Off controller and it was found that the ON/OFF controller did not consider the thermal inertia of the building, which resulted in an increase in the indoor air temperature. The above study was only limited to electrical heating systems and was applied on a test cell. Argiriou et al. (2004) and Kanarachos and Geramanis (1998) used the prediction capabilities of ANN, and used ANN as a controller for hydronic heating systems. Kanarachos and Geramanis (1998) implemented ANN controller on a single zone hydronic heating systems, whereas Argiriou et al. (2004) tested their controller on an office building. Both of these works considered the same type of heating system. However, Kanarachos and Geramanis (1998) did not perform prediction of indoor temperature and outdoor weather conditions. The ANN controller designed in Argiriou et al. (2004) was not able to cope with the cold Monday mornings due to the late start of the heating system and resulted in higher values of discomfort. This problem was tackled by using another ANN module to cope with step changes in the set-point temperature.
PID controller is a most widely controller for HVAC system but they cannot deal with the non-linearities present in HVAC systems. Curtiss et al. (1994) used ANNs for local and global control of a commercial building's HVAC system to address the problems of PID controller. For local control a hot water coil was considered, whereas for global control whole HVAC system was considered. From results of local control, the non-linear behaviour was clearly shown, and the valve exhibited critically damped oscillatory behaviour at higher set-point temperature. It was also shown that PID results are very much dependent on the gain values. The system became unstable at a gain value higher than 2. ANN results showed quick response and minimal overshoot. Curtiss et al. (1994) also studied the effect of learning rate on the controller's results. For global control, the ANN was used to model the system (predicting its power consumption). The proposed global control method consisted of two ANNs. The first ANN was used to learn the relationship between different input variables, weather conditions and power consumption. The second ANN was used to find optimal operating conditions by adjusting local loop setpoints. In order to simplify results, only cooling mode was studied. ANN outperformed PID controller in controlling heating coil and also showed promising results in controlling the whole HVAC system. ANN controller is more suitable for non-linear problems, and this study clearly showed that it performed better than the PID controller, which was unable to consider the non-linear behaviour of HVAC components.
An NN-based optimization method was developed by Ning and Zaheeruddin (2010). The method integrated an NN-based optimization technique with a model-based prediction. The objective of the research was to find optimal set-points for a variable air volume (VAV) system. The authors used two different zones: in one zone the cooling load was mainly affected by internal gains, whereas in the second zone the cooling load was mainly affected by the outdoor air conditions. The results showed that both the night reset strategy and ANN control maintained indoor air temperature at the desired value. However, the energy cost for ANN case was lower than the night reset strategy. Fan energy use was higher for ANN controller because it took advantage of the cold outdoor air temperature in the morning and tried to pre-cool the building. Simulation results were compared with the conventional night reset operation scheme and it was found that ANN based controller saved around 19% energy under part load conditions and 10% under full load conditions. Energy savings were higher for part load because the temperature set-points for night reset strategy were not closer to the optimal values. Song et al. (2003) developed a robust NN controller for a VAV system. This was the first work that addressed the stability and robustness issues of a VAV system by using a robust NN controller. A normalised learning algorithm was used by the authors to train the NN. This method eliminates the requirement of a bounded regression system. One of the limitations of ANN controllers is that their model is dependent on historical data and any changes in the operating condition can result in invalidity of the model. The authors used an adaptive dead-zone scheme to enhance stability and adaptive capabilities of the system. The controller was developed to deal with system's uncertainties and disturbances because of incomplete knowledge of the system. When compared with standard NN, the proposed controller performed better. Simulation results showed that the robust NN controller guaranteed convergence and smaller error as compared to the standard back propagation neural network. Yuce et al. (2014) proposed an ANN based real-time prediction engine for energy consumption and thermal comfort of an HVAC system for an indoor swimming pool. They utilised two control variables and six environmental variables as inputs to predict thermal energy consumption, electricity consumption and thermal comfort level of the swimming pool. To train the ANN, they used a calibrated MATLAB Simulink-based thermal model to generate the data set. The authors tested several different ANN topologies to find the best configuration, and it was found that the network with Levenberg-Marquart as training algorithm generated better results with two hidden layers. The authors used ANN for developing the model of the system, further work towards developing an ANN controller needs to be done.
General regression neural network (GRNN) were also used for HVAC control because of its quick learning ability, fast convergence and easy tuning as compared to the standard back propagation NN. GRNN was used by Ben-Nakhi and Mahmoud (2002) to optimize air conditioning setback scheduling and to demonstrate its advantages over standard NN. The objective was to predict the time of the end of thermostat setback while ensuring that the indoor design temperature is restored at the start of business hours. After office hours the indoor air temperature was allowed to rise to save energy. Two public buildings were considered in this study, and ESP-r tool was used to generate a database for training and testing ANNs. The authors also used Net-Perfect algorithm to prevent the possibility of overtraining. Different neural networks were compared with each other and GRNN. The R 2 values for standard ANN were increased by increasing the number of input neurons, but this also increased the complexity of the problem. It was concluded that GRNN is a powerful technique for setback scheduling and it also performed better, even when it was introduced to the "production data" (it is the data which is outside the training data set).

HVAC fault detection
Artificial neural networks were also used for fault detection and diagnosis. They are trained on residual patterns or fault patterns to identify different faults in HVAC systems. Lee et al. (1996Lee et al. ( , 1997 used residual patterns to train ANN for fault diagnosis in an AHU, whereas Li et al. (1996) used fault patterns to identify faults. Other studies include, Wang and Chen (2002) trained ANN to diagnose faults of different sensors of an HVAC system. A GRNN was used by Lee et al. (2004) to find faults in an AHU system. Morisot and Marchio (1999) used an ANN for fault diagnosis and detection (FDD) on a VAV system. The authors mentioned the importance of using real data for training ANN. However, due to unavailability of real data, the training was performed by using a physical model. Results were presented for normal behaviour and on faulty operations. It was found that the faults were successfully detected by the ANNs. Lee et al. (1996) also applied ANN to the problem of fault diagnosis in an AHU. Eight different faults were studied, and the ANN was trained by using relationships between faults and their dominating symptoms. This approach was successfully applied to identify 8 different types of faults. The authors also used ANN for modelling and used IF THEN reasoning to construct a pattern of dominating training residuals for each fault. The number of faults can increase with the complexity of a system, this issue was highlighted by the authors, and it was suggested that it may be desirable to develop separate ANNs for different subsystems. This study was conducted under experimental conditions and its application on a real system with noise was not evaluated, which can be a limitation of this study. This study was based on steady state conditions but the problem can change when dynamic conditions are considered. Wang and Chen (2002) presented a supervisory control scheme to adapt to the presence of faults in an outdoor airflow rate control. The difference between NN output and the measurement of outdoor/supply airflow sensor is used to regain the outdoor airflow control. The proposed strategy was tested by using dynamic thermal simulations and the controller was able to maintain the indoor air quality at a satisfactory level without any increase in energy consumption. Lee et al. (2004) used another type of neural network known as a general regression neural network (GRNN) to diagnose faults at a subsystem level. The parallel structure of GRNN makes it a better option for real-time applications. IF-THEN rules were used for the classification of different faults on a subsystem level. The GRNN model was accurate in finding different faults. Li et al. (1996) developed an ANN based fault diagnosis method for six different faults. They used mathematical modelling to obtain a database to train and test a neural network. Seven operating modes (classes) were studied: normal, bad combustion, heat exchanger (for dirtiness and scale formulation), heating curve (bad tuning of radiators), leaky valve, early boost and late boost classes. Initially, only one neural network was used to differentiate between all seven classes. However, it was found that there was a risk of normal class to be detected as a control system fault. The authors used water supply temperature to discriminate heating curve class from others, which is dependent on the type of building and system. It was noted that it was difficult to find a reference value for supply water temperature that can represent good tuning of the heating curve for different systems and buildings. Different neural network architectures were tried including a combination of two neural networks. The first ANN (ANN1) was used to discriminate heating curve class from other classes and second ANN (ANN2) was used for discriminating other classes from heating curve class. It was found that on high solar radiation days, the ANN1 was unable to detect a heating curve that is too low. This ANN also, classified boost heating too late as a too low heating curve. ANN2 showed non-classification rate between 0 and 12% for different classes. Most of the bad classification was due to the confusion between early, normal and late boosts. Due to these limitations, the authors decided to use a single artificial neural network structure that can combine all fault classes instead of using multi neural networks for different classes.
A single artificial neural network architecture was used by Li et al. (1997) to solve the same problem. It was found that heating curve and boosting curve detection was easier and the correct classification rate varied from 91% to 100%. This study shows the importance of network architecture for ANN performance. The performance of single artificial neural network could be better because single network learns a global knowledge more easily than a multiple ANN (Li et al. 1997).
It is a well-known fact that the quality of artificial neural networks results strongly depends on the quality of training data and any uncertainty in the data can decrease the efficiency of a neural network. This problem was tackled by Du et al. (2009) by using wavelet neural networks. Wavelet analysis was used to process the original data and to seize valuable information. Through this method, noise in the training data can be removed and the ANNs can be easily and well trained. By constructing different data groups using essential relations and models, and combining wavelet analysis with a neural network, it was found that the ability of neural networks to diagnose various faults can be improved.
Artificial neural networks were also used by Wang and Chen (2002) for fault-tolerant control in an outdoor airflow control. A supervisory control scheme was proposed for this purpose, and sensor based demand scheme was used. The faults were diagnosed by using the residuals between the measurements of flow sensors and the outputs of the neural network. The simulation results showed that the developed strategy was able to find a good compromise between energy consumption and indoor air quality. Peitsman and Bakker (1996) applied black box models for generating models for fault detection in HVAC systems. Multiple input single output (MISO) ARX (autoregressive with exogenous input) and ANN models were used for this purpose. The examined system was fully equipped with different type of sensors and meters and therefore, it was easy to create models from the collected data. Two types of models were created with the measured data i.e. system models and component models. ANN models performed slightly better than the ARX model. This work was further extended by Peitsman and Soethout (1997) and applied ARX models for real-time fault detection and diagnosis in HVAC systems.

Discussion
Artificial neural networks are mainly used in literature due to their ability of solving non-linear problems. Artificial neural network is one of the widely used techniques to make predictions without having any knowledge of the system. ANN can be classified as data driven method and like other data driven methods, they heavily rely on the quality of the training data. It is also found that most of the studies are based on static model prediction. The model is built by using historical data and when the new data is available or when the data outside the training set is present then the previous model may no longer be valid, which limits the use of ANNs. This problem can be solved by using an adaptive ANN model: a model that can adapt to any changes in the incoming data patterns and has an inherent self-revision capability. An example of such method can be found in Yang et al. (2005), the authors used adaptive ANN models to predict building energy.
ANN fault detection and diagnostic methods are based on residual values, which depend on a comparison of actual value and value obtained from the model. Therefore, modelling error is critical in these applications. In literature, only one fault at a time was detected, and no study was found that dealt with a combination of faults at the same time. In future this problem needs to be addressed. One of the main limitation of artificial neural network is to find an appropriate network, which is a trial and error process. By increasing the number of hidden layers and neurons can minimise the error but can also increase the training time. Too many hidden layers can also result in overfitting, which in return can cause poor generalisation of ANN model.
Most of the previous studies are either carried on experimental set ups or are based on simulations, more studies should be performed on real operating systems to evaluate the performance of ANNs in real systems. In future, generalisation of ANN models needs to be addressed so that one model can be used for different buildings and HVAC systems. Different prepossessing techniques need to be investigated to improve the quality of training data e.g. use of wavelet neural networks for seizing valuable information (Chen et al. 2006;Du et al. 2009). Table 3 summarizes research work focussing on artificial neural networks.

Principal component analysis
Principal component analysis (PCA) is a multivariate analysis method (Jackson 2005;Jolliffe 2005), which is also used as a dimensional reduction technique. PCA produces a lower dimensional representation that preserves the correlation structure between the process variables and is optimal in terms of capturing the variability in the data (Russell et al. 2012). In PCA, the original variables are represented by a smaller number of components (principal components) because of the redundancy of the variables. Therefore, instead of analysing all involved variables, the PCA method focuses on analysing principal components (Wang and Xiao 2004b In Eq. (9) U T is the loading matrix, ( ) m ḿ Î U U R , which is used to assign weights to each variable. The columns of the matrix U are the eigenvectors of the covariance matrix of the variables, . In PCA, the only retained eigenvectors are the one that are associated with the first k largest eigenvalues. These retained eigenvectors represent the directions of the most variance of a system. If only k number of principal components are used, an estimation of x in Eq. (9) can be reproduced, and hence, a new sample x new can be divided into two parts i.e. estimation of ( ) new new x x and residual (e) (Wang and Xiao 2004b). In most of the fault detection and diagnosis application, Q statistics (the squared sum of the residual) or squared prediction error (SPE) is used as an index of faulty conditions. The Q statistics can be represented by the following equation (Wang and Xiao 2004b): Wang and Cui (2005) applied an online strategy to detect, diagnose and validate sensor faults in centrifugal chillers by using principal component analysis. The authors used Q-statistics to detect and Q-contribution plot to diagnose the sensor faults. They developed and trained PCA models by using three steps: decomposing of the covariance matrix of training matrixes, retaining loading vectors, and determination of Q-statistics. They developed two PCA models: one concerning energy balance and the second concerning energy performance. Tests were performed by adding bias to the measurements of different sensors and artificially corrupting their readings. The results showed that the sensor faults were successfully detected and PCA-based methodology accurately estimated most of the introduced sensor biases. The PCA methodology captured the relationship between major measured variables in the centrifugal chiller. Same kind of methodology was also applied by Hu et al. (2012) but they employed a self adaptive process to automatically remove error sampled in the original data set to improve fault detection efficiency. The authors compared the proposed methodology with the normal PCA method, and it was found that the proposed methodology significantly enhance the fault detection efficiency. An improved principal component analysis with joint angle analysis (JAA) was also used to detect and diagnose fixed and drifting biases of sensors in variable air volume (VAV) systems (Du et al. 2007a). The authors used squared prediction error plot based on PCA to detect the sensor biases and then instead of using conventional contribution plot, they used JAA plot for diagnosing faults. The authors also used PCA in another study for fault detection but used Fisher discriminant analysis (FDA), a linear dimensionality reduction technique, to diagnose fault score (Du et al. 2007b). Due to PCA's pure data-driven nature, additional methods (e.g. Fisher discriminant analysis) need to be integrated to isolate HVAC sensor faults. Wang and Qin (2005) used PCA models at both system and terminal levels of a VAV system to detect and diagnose sensor faults. The authors used T 2 statistic and square prediction error (SPE) to detect faults and then these faults were isolated using the SPE contribution plot. Wang and Xiao (2004a) also reported a PCA method based fault detection and diagnosis strategy for sensor faults in a typical air handling unit. The faults were detected by either using Q-statistics or squared prediction error (SPE) and then were isolated by using Q-contribution and SPE plots, which were also supplemented by some expert rules. As most of the HVAC processes are non-linear in nature, the authors used multiple models to improve the fault isolation ability of the proposed method. The authors also tested the proposed method using measurements from the existing BMS (building management system) of an AHU in another study (Wang and Xiao 2004b). It was concluded that PCA models are able to generate useful residuals for sensor FDD and PCA-based strategies can be improved by combining them with other simple physical reasoning methods. Same kind of study was also applied by Wang et al. (2010), for fault detection, diagnosis and estimation purposes of HVAC sub-systems involving sensors faults at the system level.

Bayesian networks
Bayesian networks are probabilistic graphical models that represent relationships of probabilistic dependence within a group of variables through a direct acyclic graph (Zhao et al. 2013b). Bayesian theorem is used for calculating conditional probabilities. To demonstrate this, let's assume there are two random events: X and Y, and the probability of Y is greater than zero. If we have a probability of event Y, the probability of the event X, represented by ( ) P X Y , is given by Eq. (11): where, P(XY) is the joint probability and can be found using Eq. (12): Assuming Y 1 ,Y 2 ,...,Y n , are a set of random variables and satisfy conditions: S is a certain event; (iii) they are mutually exclusive (Xu 2012). For any given event X, the marginal probability of X is given by Eq. (13): Bayesian theorem can be obtained by the conditional and marginal probabilities: The items on the right hand side of Eq. (14) are called prior probabilities and the items on the left hand side are called the posterior probability. Bayesian probabilities provide a method to calculate the posterior probability from the prior probabilities, which is the basic idea of the Bayesian inference. For a simple case, if the prior probability of fault Y i and the conditional probability of the symptom X given Y i are known (which can be found from historical data etc.) then the posterior probability can be calculated by using Eq. (14).

Bayesian networks application
Zhao et al. (2013b) developed a three-layer Bayesian network to diagnose chiller faults. The authors used probability analysis and graph theory to calculate posterior probabilities of the faults. Bayesian network is an effective and efficient method for fault detection and diagnosis under uncertain, conflicting and incomplete information. The development of Bayesian network for this study required to estimate parameters (which represented the quantitative probabilistic relationship between layers) and it was acknowledged in the study that obtaining these parameters are difficult, and further work needs to be done to reduce this difficulty. Same kind of methodology was also employed by Zhao et al. (2015) and Xiao et al. (2014) but for diagnosing faults in air handling units and variable air volume systems respectively.
Naja. et al. (2012) proposed a static Bayesian network for fault diagnostics in AHUs. The proposed method was based on analysing observed behaviour and comparing these observed behaviours with behavioural patterns that were generated under faulty conditions. The aim of the paper was to develop an FDD method that is more flexible in terms of measurement constraints (e.g. measurements that are not easily accessible/measurable in real world applications) and less dependent on model accuracy. The authors demonstrated that such problems can be formulated as a posterior estimation problems of a Bayesian model. The research needs to be extended to tackle complexity, which can result due to increase in number of air handling units.

Clustering
Clustering is one of the most popular unsupervised approaches in the area of the data analysis to group a given pattern into a meaningful sub-patterns. The clustering techniques do not utilise a supervised approach which means the given row pattern does not contain any labelled or prior information. Thus the grouping (classification) process is based on the utilised metrics such as a statistical metric (Jain et al. 1999). To group the data set according to similarity, cluster techniques employ three fundamental steps: feature extraction, determination of the similarity of the inter-pattern and grouping. The most popular clustering algorithms in literature are K-means and fuzzy-C-means (Velmurugan 2014). Moreover other computational intelligence techniques such as ANN, ensemble rapid centroid estimation and Markov chains are also widely utilised.
The clustering (grouping) process is based on the minimising the sum square distance between data and cluster centres, which is based on the Euclidean distance calculation, called K-means. For a given set of inputs, X= X1,X2,...,Xn, it is given by Eq. (15): where, Clustered is the sum square error for a given pattern, X i is the individual pattern and Cen j is the center of the cluster j. Further, the extension of the K-means by using the fuzzy membership values for the distance calculation is called fuzzy-C-mean, given in Eqs. (16) and (17) where, Clustered is the sum square error for the given pattern, X i is the individual pattern, Cen j is the centre of the cluster j, and k ij μ membership value of the k-th membership function for the i-th pattern and the j-th cluster centre. Yuwono et al. (2015) proposed a consensus clustering algorithm with an unsupervised selection to detect some unknown features in the collected data from the HVAC system to avoid the redundancy and detect the faults. The proposed method utilises ensemble rapid centroid estimation (ERCE) approach using the relative entropy to detect the faults according to the features' frequency. The proposed algorithm is utilised on the forty-nine different types of HVAC faults from the ASHRAE-1312-RP, and the results are compared to K-means features selections. This consensusbased clustering method utilises the consensus matrix that includes the individual probability information about their correspondent class centre. This information has been modelled very similar to the fuzzy-C-means membership weights to determine the cluster centres. The cluster centre info then is utilised to partition the data set into subsets using the entropy distance approach using a PSO based search process for each individual. The authors also used a non-linear auto-regressive neural network to classify the features. To compare the performance of the clustering techniques, the normalised mutual information method is utilised for the selected features. According to the results, the proposed method performed better than K-means.

Pattern matching
Pattern matching is one of the most popular technique in the area of the computational system. The idea of the pattern matching is to search for a pattern in the existing greater patterns, which is based on the similarity search (Manfaat et al. 1996). Pattern matching is highly popular in several domains including image processing, text processing, data mining and signal processing. Several techniques are available to match a pattern and determine the highest resembled pattern (class) such as ANN, fuzzy logic, PCA, tree-match, and Markov chain (Maitrey et al. 2014). Cho et al. (2005) proposed transient pattern analysis based approach to detect faults in HVAC systems. The authors generated several fault conditions by changing outputs by 20% from their normal setting, and reclassified as faulty condition and fault-free conditions. They experimentally tested the proposed method in an environmental chamber's test room. The results showed that the time evolution of fault patterns can be classified as slow and fast patterns. It was concluded that HVAC systems experiencing a fault required approx. 60 min to reach steady state, and without considering the transient behaviour, it may generate different diagnosis results.

Discussion
PCA-based methods like other multivariate statistical methods do not require full information of the system and can be trained by using historical data. These methods are also popular due to their conceptual simplicity. PCA-based methods assume that the involved variables are linear and Gaussian distributed, and are mostly suited for linear system as they look for a linear combination of original features. As HVAC systems are highly non-linear, therefore the effectiveness of PCA-based methods for fault detection and diagnosis of HVAC systems can be limited. PCA-based methods' performance also reduces with increase in the range of variables. Another limitation of PCA-based methods is that the models are time invariant, whereas the processes themselves are time-varying. Therefore, to tackle this problem, the PCA models needs to be recursively updated. The update should include: PCs including a number of components to be retained, the confidence limits for T 2 and Q, and mean covariance (Venkatasubramanian et al. 2003). Another drawback of PCA-based methods is that they do not have an ability for the future prediction. However, they reduce the dimensions of the data, which is useful while dealing with a complex large number of data set, by using a correlation matrix without prior information (Gorsuch 1988).
Bayesian networks, a class of probabilistic models, have proven ability to be effective methods for fault detection and diagnosis in HVAC systems. The method is a data-driven process that is highly dependent on the existence of the data. The larger volume of data provides a generalised solution to the probability value of the relationship between the variables. Bayesian network can be utilised on most computational systems as a predictor. However, the method has a weakness for latent node networks (Williamson et al. 2000). These weakness needs to be evaluated, and a correlation between lateral nodes and their outputs needs to be used so that the structure of the nodes can be reconstructed. Bayesian networks are highly dependent on the dataset, and noisy dataset may reduce the accuracy of the model. House et al. (1999) published their work on the application of several classification algorithms for fault detection and diagnosis, including artificial neural network, rule-based, k-nearest prototype, k-nearest neighbor and Bayes classifiers. The results showed that the Bayes classifier outperformed other methods for fault detection and had the lowest percent of incorrect diagnosis. The authors also concluded that the Bayes classifier is a simple and straightforward method that requires limited storage and computational effort as compared to other studied classifiers.
Clustering algorithms are also unsupervised classification techniques without having any prior information about the dataset. Most of the clustering algorithms utilise fixed number of classes and classify the dataset into this subclasses according to the measured distance. Mostly, the measured distance is based on the Euclidian distance, which is highly sensitive to non-linear data sets. Therefore, to avoid this weakness several hybrid statistical and intelligent techniques are integrated to increase the clustering quality. Moreover, having a constant number of classes may also reduce the quality of the clustering process (Xu and Wunsch 2005). K-means clustering technique is one of the most popular clustering algorithms, which pre-supposes the number of clusters needed to cluster the given data. This method uses all the clusters' centers so that each cluster is guaranteed to have at least one pattern. According to Venkatasubramanian et al. (2003), K-means clustering can be considered as a special case of Kohonen's clustering algorithm (Kohonen 1998), where the algorithm makes clusters in the neighbourhood to be the winner of the pattern. This can result in the problem of gravity as all the cluster centers migrate toward dense regions (Venkatasubramanian et al. 2003), which can be addressed by using fuzzy clustering technique.
Pattern matching is one of the most popular subjects in the area of the computational systems, text processing, signal processing and image processing. The technique is highly dependent on the similarity measure to determine a pattern in entire patterns. The matching process is highly dependent on the length of the match and similarity function, as both factors affect the performance of pattern matching process. Some of the similarity measures such as entropy, maximum similarity are the most widely used measures. However, the results obtained while using these measures are highly dependent on the data quality. The noisy data may reduce the quality of the similarity. Table 4 summarizes research work focussing on pattern recognitionbased methods.
6 Multi-agent systems Lavinal and Weiss (1999) defined agent as a computer system (hardware/software), located in some environment and is able to meet its objective by autonomously reacting to any changes in that environment. An agent has three basic characteristics (McArthur et al. 2007a,b): (1) Reactivity. The ability of an agent to react to any changes in its environment.
(2) Pro-activeness. Pro-activeness refers to the ability of an agent to change its behaviour dynamically to achieve its objective.
(3) Social ability. This allows an agent to communicate with other agents by using an agent communication language (ACL). This ability allows agents to converse rather than just passing data. The basic idea behind multi-agent systems (MAS) is that there are local goals rather than having overall system goal. The agents work together to perform complex tasks that are difficult to perform by a single agent. An overview of a three agents (central, local and personal) based multiagent control system is shown in Fig. 10. The system receives data from sensors and implements decisions on HVAC system. The personal agent is used to observe external environment, assist to manage occupant's information and providing feedback from other agents to their occupants. MAS has been successfully tested in homes (Joumaa et al. 2011) and office buildings (Erickson and Cerpa 2010) for managing HVAC systems, IAQ and lighting systems. In both these studies the multi-agent control system was able to learn occupancy trends and optimize building energy demand. MAS have also gained popularity in other fields of energy as well, a review on MAS applied on microgrid systems can be found in (Kulasekera et al. 2011).
MAS can be applied to large buildings with many users of different preferences. The conflict between occupants' preferences can be a challenging task, and MAS have been used to handle this problem (Lee 2010). In this study, the conflicts were resolved through communication and collaboration between object agents, and also with higher-level agents. MAS can also be used to find a trade-off between energy consumption and comfort level, which are always in conflict with each other (Kastner et al. 2010;Wang et al. 2012). In most of the studies, multi-agent systems are featured as open architecture systems, which makes them an attractive solution to be applied to different types of buildings (Yang and Wang 2013).

MAS applications
Multi-agent control systems are based on the idea of dividing a problem into small sub-problems and then solving these sub-problems; this idea is known as divide-and-conquer (Ferber 1999). Multi-agent control systems have shown great success in recent years. The following review will focus on multi-agent systems which use CI techniques to solve building energy control problems.
Occupants' behaviour has a significant effect on building energy consumption, and this research area has gained a lot of researchers' attention. The ability of MAS to divide a complex problem into small problems can be used to address energy consumption from occupant behaviour. An agent can be used to learn occupants' behaviour, Yang and Wang (2013) developed a multi-agent system for achieving better thermal and energy management in a building. It was emphasized in their research that occupant should have interaction with the building as occupants' behaviour has direct impact on the system performance and energy consumption. A personal agent was designed for this purpose and ANN was used for learning purposes. The main objective of the research was to achieve user-centered control by using  Fig. 10 Overview of a multi-agent system for building applications personal agents. The multi-agent system was designed to manage indoor environment (indoor illuminance, temperature and air quality). From simulation results, it was shown that the controller was able to control thermal environment by satisfying occupant's thermal comfort and it also reduced energy consumption. Rutishauser et al. (2005) deployed a multi-agent system for a commercial building that was equipped with effectors and sensors. The MAS was implemented by using a novel unsupervised on-line real-time learning algorithm. The knowledge of the system was represented by a set of fuzzy rules, which were used as learning algorithms. It was shown that the performance of the building was improved by using a multi-agent system. A design of a MAS for room energy savings was proposed by Bin et al. (2010). The design was consisted of four types of agents: personal, environmental, room and management agents. The authors tried to address the problem of energy wasted by switching on and off the electrical equipment e.g. HVAC system; however no simulation or experimental results were mentioned to prove authors' claim. Also, the personal agent was portable in this study, which is not a feasible solution for larger buildings because of the number of people and this can increase the initial cost of the system. Yang and Wang (2011) used a multi-agent control system for a multi-zone building. The multi-agent system consisted of three agents: central, zone and local agents. The authors integrated a particle swarm optimizer into the central agent. The central controller utilised PSO to find the best possible solution for the maximum overall comfort level inside the building. Hurtado et al. (2013) presented a multi-agent based BEMS framework to optimize energy use and to ensure minimum thermal comfort level. A multi-agent control system combined with heuristic optimization for indoor energy and comfort management was developed by Wang et al. (2012). Particle swarm optimization was used to optimize set-points and ordered weighted averaging weights (OWA), OWA aggregation was used for information fusion. A fuzzy logic controller was used to calculate the required power and PID controller was employed to control indoor environmental parameters. Simulations results showed that 3% increase in overall comfort level and approximately 9% of the total energy was saved. Mokhtar et al. (2013) proposed an ARTMAP multiagent BMS system. ARTMAP is a type of ANN that provides incremental learning inspired by how human processes memory and learns new information without necessarily forgetting the previously learned information (Carpenter et al. 1991). The advantage of using ARTMAP is that it can perform classification and prediction at the same time. This makes it a better tool for adaption of an agent as compared to the classical methods (ANN, fuzzy logic). The proposed system was applied to UCLan Samuel Lindow Building, which uses a ground source heat pump (GSHP) and gas fired boiler to meet its heating demand. The previously installed MAS BMS was underperforming and was not able to use GSHP to its maximum capability. The authors added a layer of Mediator agents in between Source and User agents' layer to categories the two energy sources into two categories. Some promising results were obtained from the simulations, and it was found that the ARtMAP based MAS with an extra mediator layer performed better than the existing MAS based BMS system.
Fuzzy membership and rules were also used by Doctor et al. (2005) to represent occupant's behaviour. Different experiments were conducted at iDorm (Holmes et al. 2002) in which agent was trained to adapt to the occupant's behaviour. From experimental results, it was found that the proposed controller when compared with genetic programming (GP), the adaptive neuro-fuzzy inference system (ANFIS) and the multilayer perceptron (MLP) neural network had less computational cost and also RMSE (root mean square error) was lower than other methods. As the proposed method was computationally less intensive, therefore it is better suited for on-line applications. Hagras et al. (2008) presented a novel agent based approach called intelligent control for energy (ICE) for energy management in a commercial building. ICE was trained to learn building's response to different variables by using different computational intelligence (CI) methods (neural network, fuzzy logic and GA). It was shown that ICE can reduce energy consumption while keeping thermal comfort to costumer-defined level. Klein et al. (2012) used MAS to improve energy and comfort management inside buildings and utilised Markov decision problems (MDP) to coordinate building system devices and occupants. The authors studied a complex problem and considered a 17-zoned three-story university building. Simulation results showed a 12% reduction in energy consumption as compared to the existing control. Liu et al. (2008) presented a multi-agent system for the existing BEMS and required input from policy management and wireless sensor network. The authors claimed that the proposed system can be adapted to most type of the buildings but future work needs to be done to evaluate the performance of the system. A back propagation ANN based MAS was developed by Liang and Du (2005). The controller was used to maintain thermal comfort at the desired level. The controller was tested on CAV and VAV systems, and it showed better performance and energy savings as compared to the conventional controllers. However, controller's implementation on large scale buildings was not discussed in the paper. This can be a serious limitation due to the convergence problem of artificial neural networks. Caraiscos (2009) used a fuzzy controller in a MAS as a behavioural system and GA to regulate the knowledge basis and membership function. Artificial neural networks minimise a cost function that satisfies the occupant's needs on an average level and, therefore, occupant does not have more participation in the system operation (Dounis and Caraiscos 2009). The authors used fuzzy logic to address this problem.

Discussion
Multi-agent systems are autonomous systems where each member system aims to maximise its own gain. MAS are utilised to implement on complex and multitask problems. Although MAS can solve these complex problems, they cannot guarantee to achieve maximum gain from the overall system. However, they offer a multitask solution methodology. Therefore, they have been utilised for HVAC problems as mentioned above. As highlighted in above section, most of the studies in the literature on HVAC system using MAS are based on the controlling set-points and learning the patterns in the buildings such as occupancy patterns. Therefore, each multi-agent system has its own topology and design such as an agent can be a learning agent or an optimization agent. Table 5 presents research work focussing on multi-agent systems.

Fuzzy logic
Fuzzy logic (FL) was developed to deal with the uncertainties that are present in real world problems. The difference between classical mathematics and fuzzy logic is that traditional mathematics requires objects to have either 0% or 100% membership, whereas fuzzy logic allows to have any degree of membership between 0% and 100% (Symans and Kelly 1999). Fuzzy logic theory mimics the human ability of reasoning and judging imprecise and uncertain problems. FL is mainly based on three modules; a fuzzifier, an interference engine and a defuzzifier as illustrated in Fig. 11. In the fuzzifier module, the non-fuzzy numbers become fuzzy, meaning that measured control inputs are converted into fuzzy linguistic values by using reasoning mechanism. The  Fig. 11 Schematic of fuzzy logic controller fuzzy inference module infers the control action for a given fuzzy input, the most commonly used inference method is the "IF-Then" rule (Pourzeynali et al. 2007), expert knowledge is often used to obtain a set of fuzzy rules. A defuzzifier is used to convert the inferred value into a crisp control value. The procedure for fuzzy logic is shown in Fig. 12.
To represent fuzzy logic mathematically, we will use fault detection process for illustration purposes, which can be considered as classification problem. Numerical input value is fuzzified by the fuzzifier to a membership of a linguistic value defined on the range of the numerical data (Lo et al. 2007). These numerical values can be represented as A={A 1 ,A 2 ,...,A N }. Each of these numerical values A k is defined by a linguistic variable that takes linguistics values from The faults in the system can be represented by the set, F={F 1 ,F 2 ,...,F M }. These faults can be classified by the fuzzy rule set, R={r 1 ,r 2 ,...,r O } and the i-th rule can be written as given in Eq. (18) (Lo et al. 2007):

Fuzzy logic applications
Fuzzy logic based systems mimic the human thinking and execution ability that helps to deal with uncertainty and vagueness (Zadeh 1965). Therefore, fuzzy systems are capable of approximating any type of problem, even with the existence of inexact information (Zheng and Xu 2004). HVAC control and monitoring problems are challenging because HVAC systems perform in dynamic, nonlinear, uncertain and multivariate environments. Therefore, a robust and intelligent solution technique is required to deal with this kind of complex problems, and fuzzy systems are one of the best candidates to handle this type of control problems (Zheng and Xu 2004). In literature, several fuzzy logic based HVAC control and fault detection systems have been proposed.

Control
PID controllers are most widely used control solutions for HVAC systems because of their simplicity. However, fuzzy logic controllers are more energy efficient, robust and also have a faster response to external disturbances because of their expert knowledge. Different authors have investigated the performance of fuzzy logic controllers for HVAC system and also compared results with PID controllers. It is often difficult to tune PID gains to their optimal values, on the other hand, sometimes fuzzy logic rules are not effective because they are linguistics rules based on human knowledge. So et al. (1997) tackled this problem by developing selflearning fuzzy logic controller and using ANN to model the AHU system. Self-learning algorithms were applied to adaptively change the crisp values during the defuzzification. Different simulations were performed, and reduction in energy consumption and faster response were achieved. However, the limitation of self-learning controllers is the requirement of real-time model of the system. Zheng and Xu (2004) used a self-regulating fuzzy controller to control an air conditioning system. The controller was based on qualitative and quantitative variables, which were used as weighting factor. The proposed controller was compared with a typical fuzzy controller, and it was shown that the self-regulating controller has shorter response time and lesser stable errors. Ali (2012) proposed a fuzzy logic based controller for air conditioning system to control compressor motor speed and fan speed to reduce energy cost and meet thermal comfort requirements. According to the result, the author utilised a low number of linguistic variables. Therefore, the fuzzification level for parameters was not smooth. Moreover, the performance analysis of the proposed method was missing. The experiment can be extended by using extra control variables such as air temperature, flow rate etc. Mongkolwongrojn and Sarawit (2005)  based controller for an air conditioning system to maintain the indoor temperature and humidity by controlling the compressor speed, heater and supply airflow rate. The proposed method generated 1 °C temperature and 2% relative humidity errors under steady state conditions. To decrease the error rate, the number of linguistic variables can be increased. Moreover, the proposed model can be extended by feeding the error back to system. A self-tuning fuzzy PI controller for HVAC system was developed by Pal and Mudi (2008). The controller was used to control the supply air pressure in an HVAC system. The fuzzy rule based system was defined according to the error change and error of the control variable. These rules were then used to adjust the output scaling factor of fuzzy PI controller. The self-tuning fuzzy PI controller (STFPIC) was aimed to overcome the process parameters variations. According to the authors, the proposed method performed well both under normal and model variation conditions. It was found that the proposed method performed better than the PID and adaptive neuro-fuzzy controllers. These experiments may need to be replicated to illustrate the consistency of the results. Ghiaus (2001) developed and tested a Sugeno-type fuzzy model and controller for a fan coil unit. The Sugeno-fuzzy system is a non-linear function obtained by interpolating between linear systems. Results showed that the PID controller had a larger settling time (225 s) compared to the fuzzy logic controller (100 s). Both controllers were able to reject external disturbances caused by the outdoor air temperature. Chu et al. (2005) proposed a least enthalpy estimator (LEE) based fuzzy control system for a fan coil unit. The controller integrated the concept of thermal comfort with the theory of enthalpy. It was shown from the experimental results that the LEE based fuzzy controller can achieve thermal comfort and energy savings at the same time. The controller allowed the room temperature to rise to bring humidity to the desired level. A Takagi-Sugeno fuzzy worward (TSFF) controller was proposed by Homod et al. (2012). This controller was explored due to their ability to speed up system response and also to reduce overshoot (Homod et al. 2012). The controller was compared with conventional Takagi-Sugeno fuzzy and hybrid cascade controllers. It was demonstrated that the TSFF controller performed better and was also more robust than the other controllers. Ahmed et al. (2007) developed a fuzzy logic based control scheme to maintain temperature and humidity of an occupied space that was served by a central air conditioning unit. The deviations of actual temperature and humidity from their desired values were used to decide the fuzzy qualifier. The control scheme was tested on two laboratory spaces that were served by the same AHU. From results, it was demonstrated that the fuzzy logic controller saved energy consumption and was also simple to implement. Tianyi et al. (2011) developed a duty ratio fuzzy controller for fan coil units. The basic concept behind the proposed controller was to fully utilise the dehumidifying and cooling capacities of a fan coil unit when the control valve was closed. The controller was implemented on a test rig of a VAV system. When compared with a conventional controller, a total energy saving of 30% was achieved by using a duty ratio fuzzy controller. Kolokotsa et al. (2005b) used average absolute error between the actual and predicted values of sensors as fault detection criterion. The system was controlled using a fuzzy logic controller, and the sensor data was collected from the BEMS to use for fault detection. The study focussed on three faults, i.e., temperature, CO2 and illuminance sensors. From results, it was shown that the fault detection system performed satisfactorily. However, its performance could have been improved by minimising the influence of external disturbances. In most of the cases, the fault was not detected due to poor prediction data.

Fault detection
A fuzzy model based approach for fault detection and diagnosis was proposed by Dexter and Benouarets (1997). A set of fuzzy reference models was used for faulty and normal operating conditions. For fault diagnosis, a fuzzy matching based classifier was used. The authors used Dempster's rule of combination to combine new evidences with previously collected ones. Fuzzy reference models were compared with each other to account for any uncertainty that may arise due to similar results. Simulation and experimental results showed that the proposed method was capable of identifying faults in a cooling coil system of an air handling unit. This method was computationally complex, which limits its application on real-time and low-cost hardware, and future work was suggested by the authors to reduce its complexity. Ngo and Dexter (1998) implemented a tool for remotely commissioning a cooling coil of an office building. The faults were detected by a fuzzy logic model based fault detection scheme, which was using generic reference models to describe faults or normal conditions. Soyguder and Alli (2010) proposed a fuzzy adaptive controller to generate the optimal proportion, integral and derivation values for PID controllers of two actuators position (damper gap rates) in an HVAC system. The first damper gate was controlled by utilising the temperature of indoor air volume. The second damper gate was controlled by the humidity of the same indoor air volume. The control variables adaptively adjusted the flow rate according to the error of the PID system. According to the experiments, the proposed method reduced the ambient temperature from 31.4 °C to 25.5 °C in 10 minutes. Moreover, the ambient humidity was adjusted from 24% to 41% in 6 minutes. However, the authors did not discuss a detailed comparison for PID controller without fuzzy system.

Discussion
Fuzzy logic has been widely used in literature due to its ability to map real-world problem with non-linear functions (Alcalá et al. 2003). Therefore, these controllers can be utilised instead of any classical controllers to achieve better performance. In most cases, the human experience and non-linearity without mathematical modelling are highly required for control logic of the problems. However, it is a challenging task to generate rules for a fuzzy logic system for every problem. Therefore, experts knowledge and best practices are used to create initial rule base. However, this knowledge may not be enough to define the rule based system. Therefore, Alcalá et al. (2006) proposed a genetic algorithm (GA) based fuzzy system for HVAC control, where they have utilised GA as a post-processing engine for rule selection, classical tuning and lateral tuning of membership functions. According to their analysis, hybrid approach has yielded much better results than the classical control system. Table 6 summarizes research work focussing on fuzzy logic systems.

Hybrid applications
In literature, several computational intelligence techniques have been proposed for controlling, optimizing and fault diagnosis HVAC systems. However, No Free Lunch theorem (NFL) (Wolpert and Macready 1997) states that there is not a single technique to solve all type of problems. This can also be related to the complex nature of HVAC systems. Therefore, several hybrid techniques have been proposed to solve, control and diagnose in the HVAC systems, including artificial neural network, genetic algorithm, fuzzy logic, particle swarm optimization, etc.
8.1 HVAC control Ursu et al. (2013) analytically modelled an HVAC system and then used a Fuzzy Supervised Neuro-Control based intelligent controller. The controller had two components: neuro-control and fuzzy logic control. The neuro-controller was utilised to generate the volumetric airflow rate and the water flow rate of the chiller/heater. The fuzzy logic controller was used to supervise the neuro-controller to counteract the saturation. The neuro-controller was a single layered artificial neural network (ANN) and the proposed fuzzy logic controller was a Mamdani type. The fuzzy controller was activated when the neuro-controller saturated. The usage of single neurons might have reduced the performance of the neural network, and a multi-layered ANN might have performed better. The processing time required for multilayered NN can be a drawback for this type of controllers. An optimal intelligent controller by using fuzzy controller and particle swarm optimization (PSO) can be used for HVAC systems (Khooban et al. 2012). The aim of the study was to control the air supply pressure of an HVAC system. Input membership function, first order Sugeno type polynomial functions and the coefficients of the PID controller were optimized simultaneously by using random inertia weight based PSO. The controller was compared with PID, adaptive neuro fuzzy (ANF) and self-tuning fuzzy PI controller (STFPIC). The performance of the proposed controller was better than the other controller and also its peak overshoot rate was lower. The proposed optimal intelligent control is an efficient technique for HVAC air supply pressure controller. Hadjiski et al. (2007) proposed a hybrid multi-agent system (MAS), dynamic ontology (DO) and ant colony optimization (ACO) technique to control an HVAC system. The proposed method was a combination of data driven and knowledge driven methods to improve system's stability, speed, internal communication rate and robustness. ACO based optimal control received the information from MAS then evaluate the probability of the decision to update the DO. The ACO mechanism chose an optimal control according to the probability of decision quality. Moreover, the knowledge changes in agent system were used to eliminate bad decisions. The proposed method only presented the hybridization of MAS, DO and ACO but was not implemented on multidimensional HVAC system on local and system level. Chow et al. (2001) used an ANN-GA based chiller system optimization system to reduce the cost of electrical and fuel energy usage. The aim was to find optimum values of chilled water, cooling flow rate and temperature set-points for cooling loads. Both single and two hidden layered topologies and up to fifteen nodes per layer were tested. Levenberg-Marquardt based training algorithm with tangent-sigmoid (hidden layer) and linear (output layer) transfer functions was used as training algorithms. The authors utilised this trained ANN as a simulation engine to generate outputs for GA. Although the methodology is well defined, the performance of GA-ANN was not illustrated. Moreover, ANN tuning can be extended including other training algorithms and transfer functions.
Yang and Wang (2012b) used multi-objective particle swarm optimization (MOPSO) to find an optimized solution between energy and comfort. A multi-agent control system was proposed, which consisted of two primary categories of agents (central coordinator agent and local controller agents). The central agent was responsible for cooperating with the optimizer to ensure occupants' thermal comfort should meet the desired criteria. The local controller agents were used to control temperature, illuminance and CO2 concentration. In this research, there was no information about building performance which may limit its application. Wang et al. (2011) also used a MAS with PSO for energy and comfort management. The framework consisted of central coordinate agent, local controller agents (for controlling thermal and visual comfort and air quality) and load agents.
In most of the applications in buildings, ANNs are used for prediction purposes. Ferreira et al. (2012) presented results on neural network based predictive control. ANN was used to predict predicted mean vote (PMV) and MOGA was used to find an optimal number of neurons for ANN. The proposed method resulted in energy savings of more than 50%. Nassif (2012) also used ANNs to model an HVAC system and then the system operation was optimized by using a genetic algorithm. An NN predictor for indoor air temperature was used by Marvuglia et al. (2014). The prediction was used for the fuzzy logic controller to regulate thermal comfort inside an office building. Lo et al. (2007) proposed an automatic fault detection and diagnosis system using fuzzy-genetic algorithm for an HVAC system. Faults were represented at different levels of the monitoring system and recognised using fuzzy-genetic system. Genetic algorithm (GA) was utilised to train fuzzy rule using simulated data generated from HVACSIM+ simulation program, and a fuzzy system was utilised for evaluating the fitness of individuals of the GA's population. The results showed that the proposed method performed better with a higher number of GA populations. The accuracy of the system could have been increased by using more linguistic variables in fuzzy logic instead of three (i.e. positive big, zero and negative big). Fan et al. (2010) proposed a hybrid self-adaptive fault detection and diagnosis system using artificial neural network (ANN), fuzzy c-means (FCM) and wavelet analysis (WA) for air handling unit (AHU). The proposed method has two main stages: the first stage consisted of two back propagation neural networks (BPNN) for fault detection and the second stage consisted of a fault diagnosis stage utilising WA, FCM and Elman neural network (ENN). In the first stage, BPNN was trained by using normal system's operating data and a sensitivity analysis was implemented on the input variables to determine the most significant inputs. The significant inputs were selected for the second BPNN for fault detection. In the second stage, the WA was used to extract the approximation coefficient. This information then was utilised to develop an ENN for diagnosis of sensor faults in the AHU. The authors also used FCM to cluster the approximation coefficient and to determine the cluster centres. The proposed model was sensitive to the threshold during the fault detection, which affects the accuracy of the stage. Moreover, it was recommended to keep the threshold to 1% of the level and therefore the model needs more robust approach. Dehestani et al. (2013) proposed an artificial neural network (ANN) and online support vector machines (SVM) based fault detection and isolation approach for HVAC system. The aim of the study was to reduce the maintenance cost and increase the utilisation of the HVAC system using a minimum number of data set. An ANN was used to generate a reference model for SVM and then SVM was utilised to detect faults in the HVAC system in accordance to a reference model both for online and offline conditions. In this study, the detailed information for ANN and the methodology of the sensitivity analysis and accuracy of the process were not well defined. A fuzzy logic and artificial neural network based fuzzy-neuro health monitoring system for HVAC control systems was developed by Allen and Rubaai (2013). The aim of this study was to detect the abnormal operating conditions and generate fault signatures in the HVAC system using fuzzy logic and then to classify the fault signatures using ANN. The proposed method detected all the normal conditions correctly using a fuzzy controller, and there was no error generated during the classification stage. On the other hand, ANN misclassified one of the four faults. Du et al. (2014b) developed a subtractive clustering approach for classification of sensor faults in the HVAC systems. The authors used a combined neural network (combination of basic and auxiliary neural networks) to detect fault and then used clustering analysis to diagnose the fault sources. The combined neural network was used to overcome the complexity of patterns in the HVAC systems. Among other clustering techniques, the authors used adaptive subtractive clustering analysis because of its adaptive classification capacity. When the neural network is to detect a fault or receive any signal, the subtractive clustering algorithm search the nearest cluster centre for the received pattern, whereas the cluster centre is calculated according to the historical data density information. It is believed that with the integration of dual neural networks, the detention efficiency of false alarm, detection time and missing alarm will be improved.

Fault detection
Li and Wen (2014) presented a hybrid fault detection and diagnosis strategy based on PCA and Pattern Matching methods. The pattern matching method was used to locate period of operation from a historical data set whose operational condition is similar to target operating conditions and then building PCA models for these identified periods. The authors used the Mahalanobis distance measurement approach as it is less sensitive as compared to Euclidian based distance measurement for the scale and correlation inherited in the data sets. It is evident from previously described studies, where authors used different techniques to preprocess the training data, that preprocessing can enhance the sensibility of PCA models. The same idea was utilised by the authors and used pattern matching method to preprocess the training data. The authors used this method to detect air handling unit's faults. The authors used both faulty and fault-free data to examine the proposed method. The authors did not perform online testing; however, it was believed that the proposed approach will significantly improve the sensitivity of fault detection process.

Discussion
In this section, several hybrid computational intelligence techniques have been discussed. Moreover, each individual technique and its characteristics were discussed one by one in previous sections. Each of these techniques has its own characteristic, e.g. some techniques have learning, some have control and some have optimization abilities. Some problems in HVAC system do require a combination of different techniques to overcome complexity and also to propose a better solution. With respect to the purpose of solution, these hybrid methodologies generate better solutions and offer a higher gain in overall system performance. This approach fits to the system theory, in which, the aim is to maximise the gain in overall system performance (Hanson 1995). This gain can be time and cost reduction, and induction of the comfort. HVAC systems consist of several subsystems, which have highly interdependent relationships with each other. Therefore, several hybrid techniques are presented to illustrate these complex problems and solution techniques. Table 7 summarizes research work focussing on hybrid applications.

Trend analysis
Graphical information of all the works included in this paper is presented in Fig. 13. Figure 13(a) shows the research work published in different journals and conferences. The  majority of the work was published in Energy related journals, e.g., Energy and Buildings, Building and Environment, Applied Energy, etc. Also, from the review, it is evident that most of the multi-agent research studies were presented in IEEE conferences. The works published in control related journals/conferences are lesser as compared to the work published in energy or Building related journals/conferences.
Percentage of research work in the field of CI applied to HVAC system published by the authors of different countries is shown in Fig. 13(b). USA looks to be more focussed on HVAC efficiency problems with more than 20 percentage studies conducted in the USA. In Europe, UK has shown more interest as compared to other countries. It is also evident that other developed countries/special administrative regions such as Hong Kong SAR, Singapore, etc. have also made efforts to address HVAC problems with the help of CI techniques. There are only a few studies that were conducted and published in the developing countries, which needs to be changed as developing countries have more energy related issues (e.g. load shedding) as compared to developed countries.
The most common objective function was energy consumption and 38% of the studies tried to minimise energy consumption. Thermal comfort and indoor air quality were also important cost functions and were used in 29% and 11% of the studies respectively. The researchers also considered occupant preference and visual comfort as objective functions. Other objective functions include occupant schedule, maximising renewable energy use, energy cost, cost and energy efficiencies. Multi-agent based studies considered more number of objective functions at a time as compared to other CI techniques because of their ability to divide a bigger task into a number of small tasks.
Metaheuristic algorithms, particularly GAs are more popular amongst researchers. ANNs were also used in HVAC related problems due to their ability to model non-linear problems. Back propagation was one of the researchers' preferred choices for training ANN. Among different BP algorithms, Levenberg-Marquardt back propagation was mostly used because of its fast training speed and better accuracy. Fuzzy logic was also used for control and fault detection purposes. There are a fewer number of studies that used multi-agent systems, but their use is increasing day by day.
In literature, Matlab (Mathworks 2012) has been used for about 47% (including Simulink) of the total work reviewed in this paper. Matlab has its own optimization O Fig. 13 Summary of compiled literature review toolbox, which includes artificial neural networks, fuzzy inference system, multi-objective genetic algorithm etc, and makes Matlab/Simulink an attractive option to choose. Matlab/Simulink are mainly used for designing controllers or for coupling building energy simulation engines and optimization algorithms. Hamdy et al. (2009) mentioned that besides having many useful optimization algorithms libraries, Matlab provides an opportunity to the user to use other features as well e.g. use of databases, data analysis, graphical user interface, etc. EnergyPlus (Crawley et al. 2001), an open source building simulation engine, constitutes 12% of the literature. EnergyPlus has no interface, which limits its use to some extent. EnergyPlus was mostly used for generating databases for ANN training and also used as an evaluation engine in optimization problems. TRNSYS (Klein et al. 1976) stands for TRaNsient SYstems Simulation Program and is used 19% in the literature. It was mainly used for dynamic simulations of building zones and also for generating data for ANN training. Researchers have also coupled TRNSYS with other optimization software tools.

The survey
In this article, we presented a comprehensive review discussing computational intelligence techniques for HVAC systems. In literature, several applications that use Computational Intelligence (CI) techniques can be found in the area of robotics, machine vision, computer science, mathematics, physics, chemistry and operational research. CI techniques have also several applications on HVAC systems and most of these applications are based on Genetic Algorithm (GA), Evolutionary Programming (EP), Ant Colony Optimization (ACO), Particle Swarm Optimization (PSO), Artificial Neural Network (ANN), Principal Component Analysis, Clustering, Bayesian Networks, Pattern Matching, Fuzzy Logic, Multi-Agent Systems (MAS) and hybrid combinations of these techniques. There is clearly a growing trend in the popularity of CI techniques for control and fault detection applications in HVAC systems. This is due to the evaluation in computation power available to apply these techniques on both experimental and real-life problems, and also, due to more stringent requirements for building energy efficiency and in particular for HVAC systems.
As each CI has its own merits and demerits, it is not possible to utilise one unique technique on every application. Therefore, different types of CI techniques have been found in the literature for different applications. ANNs were used due to their ability to solve non-linear problems and have been widely utilised on HVAC systems. The quality of ANN results is highly related to the quality of training data set. However, if the data is post-processed, it can produce better results for HVAC systems. HVAC problems are computationally extensive and as simulation-based optimization techniques often require hundreds or thousands of simulation runs which may make optimization techniques infeasible to apply on real-time basis. Artificial neural networks are one of the promising solution to solve this problem. However, modelling a sensitive objective function using ANNs can be a challenging task as small deviation can make a huge difference. Most of the research conducted on fuzzy logic controller also performed a comparison with the state of the art controllers (e.g. PID controllers). It is well-known fact that tuning PID gains can be a challenging task. Sometimes fuzzy logic rules are not effective as they are based on human knowledge. Fuzzy logic controllers can be energy efficient and robust controller as compared to PID controllers and also have better/faster response rate to external disturbances. Generating rules for fuzzy logic methods can be a challenging task and, therefore, best practices and experts knowledge are used to create initial rules. A clearly identified gap is the need to implement CI techniques on real systems and buildings, as most of the reviewed studies were based on virtual systems using computer simulation.
Metaheuristic algorithms are the most popular CI techniques due to their different specifics and search abilities e.g. genetic algorithms and ant colony optimization are more suited for global search whereas particle swarm optimization has a strong ability in local search. Evolutionary Algorithms (EAs) such as GA and EP are stochastic based optimization techniques offer population-based solutions and as HVAC systems being highly complex, stochastic problems can be optimized in a short time period with better quality results by using EAs. Single individual based optimization techniques look for the optimum solution that may be cost effective for a simple problem with non-complex solution space but it can be useful to solve complex problems (e.g. optimization of HVAC operation) by using populationbased techniques (e.g. genetic algorithm). Particle swarm optimization can be categorised as one of the efficient stochastic based optimization algorithms but its efficiency is highly dependent on the tuning of weights and inertia parameters. Pattern recognition-based methods have been used for FDD purposes, but their usefulness depends on the quality of the data used for generating models or patterns. Several advancements are seen in this field of research but their application on HVAC system is still lacking.
The work presented in this paper is the first of its kind that presents trends analysis of research focussing on CI techniques applied to HVAC system. It revealed that most of the developed countries are showing greater interest in energy efficient HVAC systems, as most of the reviewed studies were conducted in developed countries/SAR and low number of studies were conducted in developing or underdeveloped countries. Genetic algorithms were applied in most of the studies to solve HVAC related optimization problems. Interest in MAS is increasing due to their ability to solve the problem by dividing it into a number of small problems. Matlab/Simulink was the most widely used simulation tool followed by building energy simulation tools (EnergyPlus, TRNSYS, DOE-2 etc.).

Future directions
 HVAC design problems are different than the control problems in a way that optimization process is only required at the design stage; however optimizing control problems needs computation power throughout the HVAC running cycle. Which means, we may be saving energy by efficiently controlling/optimizing HVAC systems but on the other hand we are also consuming energy while running computationally extensive equipment. Therefore, the future research should focus on using single board computers (Raspberry Pi, Banana Pi, BeagleBoard etc.), which use less power.  The issue that may arise by using single board computers could be the use of appropriate evaluation engine. As dynamic simulation tools were the main evaluation engines used in the research studies and are computationally extensive. The use of single board computer would also mean using some form of surrogate models instead of detailed simulation models.  Reductions in initial and running costs can also be achieved by using cloud computing technologies. The emerging cloud computing technologies provide a way to optimize HVAC systems' operation by using a shared and dynamic infrastructure. The popularity of these technologies depends on the advancements in internet connection speed and further future advancements will allow us to control HVAC systems using CI techniques on a nearly real-time basis and shared IT infrastructure.  Another area that requires attention from building research community is to sufficiently explore the application of parallel processing to improve simulation results without sacrificing accuracy.  From the literature review, it is evident that it is highly desirable to combine different CI techniques (hybrid applications) to overcome their deficiencies. It is also evident that no single CI technique has all the desirable features for control, optimization or FDD system. Therefore, most of the methods can complement one another resulting in a better CI based optimization, control and/or FDD system. In most of the cases, hybrid techniques e.g. ANN-GA, Fuzzy Logic-GA or ANN-Fuzzy Logic provided better results. Therefore, not only other algorithms such as the Bees Algorithm, Artificial Bee Colony and Fish School Algorithm, Type 2 Fuzzy set etc. need to be explored for optimizing HVAC systems, future research should also focus on their hybrid applications with currently applied techniques.  The use of MAS is growing increasingly, the current MAS techniques learn from user interactions and efforts should be made to test different learning algorithms that do not require any interaction or feedback from user.  The number of articles presented in this paper is merely to demonstrate the usefulness and popularity of CI techniques in HVAC optimization problems. It is hard to conclude which of these techniques is suitable for a particular application. However, there should be future studies that may serve as a guide to select CI techniques for different applications e.g. studies comparing different CI techniques for test problems and then recommending best suited techniques for that particular problem (as we have test functions to evaluate characteristics of various optimization algorithms).  Most of the studies were validated by simulations or on small-scale HVAC systems. The practical validation needs to be performed on commercial HVAC systems and controlling them on real-time basis.