Introduction

During the second half of the twentieth century, metaheuristics have attracted the interest of scientists and engineers dealing with complex optimization applications. Research concerning algorithmic design developed in parallel to the introduction of new optimization techniques. The majority of the developed algorithms were inspired from natural or artificial procedures in order to carry out their internal search process.

Harmony search optimization algorithm

Music composition inspired (Geem et al. 2001) to introduce the harmony search algorithm (HSA). HSA imitates the way a musician plays within a music group, developing a selection process that optimizes specific problems. It belongs to the category of “neighborhood metaheuristics” that produce one possible solution per iteration. Initially, HSA was applied in urban water systems. Since then, there has been sustained and increasing interest in HSA applications, and apart from water engineering optimization problems (Kougias and Theodossiou 2013), HSA has been used on a vast variety of implementations (Manjarres et al. 2013).

Its efficiency has been extensively tested in the literature using established test functions and by comparing HSA’s performance with other competitive techniques (Das et al. 2011). HSA’s efficiency is also indicated by its popularity among those involved in the field of optimization and the rapid increase of the number of its applications.

Optimization techniques

The application of metaheuristics can be a challenging task and often requires advanced programming skills, since using metaheuristics normally involves developing ad hoc programs. Each program needs to correspond to the specifics of a single application and such an approach, apart from being time-consuming, imposes barriers to users with limited programming skills.

For this reason, a variety of optimization software in the form of user-friendly toolkits has been developed to answer the different needs. The development of such software started in the late twentieth century and ranges from programs developed in universities with an academic direction to commercial software developed in corporate environment.

State of the art analysis

In “Recent advances in optimization toolkit software” section, the authors present an overview of existing optimization toolkits, their unique characteristics, the programming language used, and their strengths—limitations. A significant outcome of “Recent advances in optimization toolkit software” section’s survey was the absence of a generic HSA-based optimization software among the existing toolkits. Thus, users with limited computing skills are currently excluded from the use of HSA. At the same time in complex applications, where consecutive runs with different parameter-setting might be needed, HSA’s use is prohibitively labor-intensive.

The present research aims to cover this gap and provide an alternative, user-friendly approach. Accordingly, an optimization toolkit applicable to a wide extent of problems related to clean technologies has been developed. The toolkit’s novel characteristic is that in its core it uses harmony search, resulting to the first HSA toolkit built, so far. In “The HSA optimization toolkit” section, we present the design goals and the unique characteristics of the developed software, highlighting its contribution in bridging the existing gap in robust optimization tools that at the same time are easy-to-use.

The HSA optimization toolkit

Design goals

The main purpose of the HSA toolkit is to overcome the aforementioned barriers and provide an interface, where users input the objective function and the constraints in normal mathematical formula, in an easy, quick, and effective way. Thus, the main driving factor behind the development of our toolkit was to programmatically support the HSA by means of a GUI-based software framework, since to date there has been no similar effort.

Another design goal is to support a variety of problems, namely by allowing the unrestricted definition of variables and constraints. This could be further supported by focusing the toolkit not only on a sole metaheuristic algorithm, but also offering a selection among variants. As mentioned in “Recent advances in optimization toolkit software” section, there is no ideal optimizer targeted at all types of problems. In this respect, the first version of our toolkit considered HSA and certain variations of HSA for optimization.

A third aim is to support the degree of responsiveness to decision-making. This is supported by the straightforward method of data input to the GUI that facilitates consecutive runs. Thus, updates and changes in the parameters of each application can be adopted in a timely manner, by altering the values of the corresponding equation constraint in the GUI. This feature is further described in “Description of the HSA optimization toolkit” section, showing the responsiveness of the developed toolkit, especially when compared to ad hoc designed software. Briefly, the design goals of the HSA optimization toolkit are

  1. (1)

    User-friendly, GUI-based optimization with HSA,

  2. (2)

    Support a variety of problems and applications,

  3. (3)

    Timely responsiveness.

Description of the HSA optimization toolkit

The toolkit was built using MATLAB to benefit from its increased performance in dealing with advanced mathematical computations. MATLAB itself supports optimization through the optimization toolbox (MathWorks 2016). Moreover, many of the research works seen in “Recent advances in optimization toolkit software” section have also utilized the facilities that MATLAB has to offer to build optimization toolkits. Accordingly, we designed and developed a software environment to define and solve optimization problems using the HSA optimization algorithm. To the best of our knowledge, this is the first effort of kind to support the HSA. The toolkit allows for parameterization of the algorithm using an interactive graphical environment, for the definition of problem constraints and of the objective function, as well as the visualization of the results.

Fig. 1
figure 1

Architecture of the MATLAB-based HSA optimization toolkit

Full size image

Figure 1 presents the architecture of the MATLAB-based HSA optimization toolkit. All functional components of the toolkit are built on and make use of MATLAB function calls for their operation. The optimization toolkit GUI is users’ entry point and the main component from which the optimization process commences. Subject to the users having entered the desired algorithmic parameters, the objective function, and problem constraints, there exist dedicated components to handle these variables. In particular, the constraint handler is in charge of parsing the constraints, while the parameter handler is responsible for parsing the parameters. Both constraints and parameters are marshaled into data objects to be passed to the algorithm engine component of the toolkit. The latter component is fundamental in invoking the user-defined HSA variation and performing the actual optimization process according to the received constraints and parameters. Assuming successful completion of the optimization process, the solution finder component collects the list of results and displays it to the user.

The user interface of the toolkit is based on the established model-view-controller (MVC) paradigm (Krasner and Pope 1988), therefore separating the internal representation of information to the actual presentation of the information to the users. As illustrated in Fig. 2, the user is given the option to modify the values for the key parameters of the HSA. Moreover, the user enters the definition for the objective function to be optimized, as well as the constraints to be taken into account during the optimization. The current implementation supports three variations of the HSA in regard to the handling of variables’ values according to constraints, out of which the user needs to select the active variation to be applied for optimization. The three variations include

HSA by choice :

In this case, both solutions stored in the memory and new candidate solutions are assigned values that strictly adhere to the constraints, thus averting the use of values that do not respect them.

HSA by penalty :

In this case, there is no restriction in considering values for the memory and every new harmony can assume values that negate the constraints. However, their evaluation as far as the objective function is concerned is subject to a penalty.

HSA by range :

In this case, the constraints are completely ignored and the different variables are assigned random values from within a range of values arbitrarily defined by the user.

Fig. 2
figure 2

Main view of the graphical user interface of the HSA optimization toolkit

Full size image

It should be noted that the toolkit also allows display and optionally store the results of an optimization process. The latest version of the HSA optimization toolkit is a software of around 7000 lines of code and requires a MATLAB compiler runtime for its execution. Various optimization tests have been used to evaluate the toolkit’s performance and the equations of the input boxes in Fig. 2 correspond to such a test. The toolkit’s quite promising performance is also indicated by the execution time both in the test-function and in the case study of “Application on renewable energy generation management” section, for which the execution time was less than 1 min.

Application on renewable energy generation management

The developed optimization toolkit has been applied in a real-world application, where the best management practices for a small-scale hydropower (SHP) station need to be identified. Since SHP’s energy output is proportional to reservoirs’ releases, water reserves are managed both seasonally and daily.

Seasonal management secures water availability in periods of low inflow, ensuring the reservoir will also serve a set of diverse functionalities (ecological conservation, energy production, irrigation-drinking water, flood mitigation, and recreation) on an intra-annual basis.

Daily management basically focuses on energy output, aiming to maximize the direct and indirect benefits of energy production, according to local conditions and policies. In the case of independent mini-grids or hybrid systems, the SHP’s optimization aims to secure continuous energy supply, mini-grid balance, and minimum amounts of spilled energy (Moner-Girona et al. 2016). SHPs interconnected to a central grid sell the produced energy for a flat or varying price.

The present research analyzes the latter, SHPs connected to the main grid, selling electricity in prices proportional to demand. The developed model aims to increase the flexibility of SHPs’ operation and fortify them against electricity price fluctuations. This is related to climate models’ projections for significant variations on future water availability that might affect hydropower wholesale electricity prices and SHPs’ economic viability.

Characteristics of the small hydropower station

The studied SHP station has a design power capacity of 125 kW, with a crossflow-type turbine. The available hydraulic head is 20 m and the design maximum discharge \(Q_\mathrm{max}=0.72\,\mathrm{m}^{3}/\mathrm{s}.\) The water flow of the rivulet flowing into the reservoir has been observed and recorded for several consecutive years, and Fig. 3 shows the monthly average of the recorded values that indicate expected monthly inflows.

Fig. 3
figure 3

Mean monthly water inflow rate to the reservoir under study

Full size image

The water discharge varies significantly throughout the year and becomes very low in summer. Naturally, the reservoir is gradually filled in the high-flow period (December–February) and contributes to the local needs in summer, when inflow is minimized (July–September). Considering that the active storage of the reservoir is \(140.000\,\mathrm{m}^{3},\) the average reservoirs’ contribution during the low-flow period is estimated \(0.02\,\mathrm{m}^{3}/\mathrm{s}.\) Respectively, during the high-flow period water is stored with an equal rate (Table 1).

Environmental flow

Among other considerations, water management needs to also concern environmental conservation. Environmental flow (EF) supports ecosystem sustainability, reflecting the water volumes the ecosystem needs for its own functionalities. Thus, the facility maintains a minimum flow in the river, that is adequate for fish population, wildlife conservation, and water quality. In a recent study (Patsialis et al. 2014), the authors estimated the uniform EF value of the studied river, according to the current legislation: \(\mathrm{EF}=0.07\,\mathrm{m}^{3}/\mathrm{s}.\)

Water availability for hydroelectric energy production

The minimum water discharge appears in September (\(Q^\mathrm{Sep}=0.05\,\mathrm{m}^{3}/\mathrm{s}\)). Comparing September’s discharge with the minimum flow, necessary for ecological conservation (\(\mathrm{EF}=0.07\,\mathrm{m}^{3}/\mathrm{s}\)), it is evident that the available water inflow can not cover the ecological needs. Accordingly, in September, the entire rivulet flow and the contribution of stored water (\(Q=0.02\,\mathrm{m}^{3}/\mathrm{s}\)) needs to be used for ecological conservation. Such water shortages often force hydropower plants to curtail electricity production.

Table 1 Monthly distribution of water availability (\(\mathrm{m}^{3}/\mathrm{s}\))
Full size table

In Table 1, the average values of monthly flow-rate availability are presented. The second column corresponds to the average rivulet’s inflow to the reservoir, while the third column illustrates the requirements for ecological conservation (EF regimes). The fourth column shows contributions to storage (negative values) or stored water releases (positive values). The fourth column present the net flow-rates for hydropower.

Energy tariff structure

The present research shows the applicability and the advantages of the developed toolkit on the optimization of the daily operation of the SHP. It maximizes the economic benefits gained by energy production, utilizing the available water quantities. In this respect, a highly important factor is the energy pricing pattern, since production needs to be maximized in peak hours and minimized when energy demand and prices are low.

Energy prices are defined in the complex context of the energy market, whose mechanisms are beyond the scope of the present research. Attempting to simulate price fluctuations, a pricing pattern that varies both hourly and seasonally (Fig. 4) has been selected. Summer tariff is applied from May to September, corresponding to the different demand patterns between summer and winter. Naturally, the toolkit can adopt various pricing patterns or incentive schemes.

Fig. 4
figure 4

Hourly fluctuation of electricity prices in summer and winter

Full size image

Water availability is identical in some months: October–April, November–March, May–July, and June–August (Table 1). Coincidentally, between these months, the same pricing policies apply; the first two pairs are subject to the winter tariff scheme and the remaining to the summer one. With water availability and pricing policies being identical, the management strategies in these months will be uniform.

Results

Model formulation

The energy production of a SHP is calculated from Eq. 1:

$$\begin{aligned} E_{t}= P_{t} \times T = \left( n_{t} \times \rho \times g \times Q_{t} \times H \times 1000\right) \times T \end{aligned},$$
(1)

where \(E_{t}\) is the produced electricity at hour t (kWh), \(P_{t}\) is the available power at hour t (kW), T is the duration of SHP’s operation (h), \(n_{t}\) is the station’s efficiency during hour t, \(\rho\) is the water density (kg/m\(^{3}\)), g is the gravitational acceleration (m/s\(^{2}\)), \(Q_{t}\) is the average water flow during hour t (m\(^{3}\)/s), and H is the hydraulic height (m).

Objective function

The aim is to attain the maximum economic benefits, derived from selling the produced electricity to the grid. This is equal to maximizing the annual product of energy output times the energy price at each time step (Eq. 2):

$$\begin{aligned} \mathrm{Benefit}=\sum \limits _{m=1}^{12}\sum \limits _{d=1}^{31}\sum \limits _{t=1}^{24} E_{t} \times \mathrm{Price}_{t} \end{aligned},$$
(2)

where \(E_{t}\) is the produced electricity at hour t (kWh), \(\mathrm{Price}_{t}\) is the electricity price at hour t (EUR/kWh), m is month, d is number of days in each month (28–31).

The decision variables in Eq. 1 are the hourly values of discharge \(Q_{t}.\) Thus, the aim of the optimization is to define the \(Q_{t}\) values throughout the day, for all months of SHP’s operation.

Constraints

The optimization process is implemented for each month with the sum defining the objective function’s value (Eq. 2). This allows fulfilling the main constraint, i.e., serving all different water needs and respecting the EF regimes. Accordingly, for each month, the daily water availability constraint is (Eq. 3):

$$\begin{aligned} V_{m}^\mathrm{SHP} \le V_{m}^\mathrm{avail.}=\left( Q_{m}^\mathrm{inflow} - Q^\mathrm{EF}_{m} \pm Q^\mathrm{stored}_{m}\right) \,*\,24\,*\,3600 \end{aligned},$$
(3)

where \(V_{m}^\mathrm{SHP}\) is the water volume directed to the SHP (\(\mathrm{m}^{3}\)), \(V_{m}^\mathrm{avail.}\) is the water volume available for the SHP (\(\mathrm{m}^{3}\)), \(Q_{m}^\mathrm{j}\) is the various water needs, as presented in Table 1 (\(\mathrm{m}^{3}/\mathrm{s}\)).

Obviously, a second constraint defines all discharge values as positive (Eq. 4):

$$\begin{aligned} Q_{t} \ge 0. \end{aligned}$$
(4)

Results: optimization using HSA toolbox

The parameters for each month of operation have been successively inserted to the HSA toolkit. That included different water flow values, the energy tariff applicable to each period, and the constraints. Several runs have been implemented and the best management practices were detected.

Fig. 5
figure 5

Hourly variation of the electricity production revenue (best practice)

Full size image

The obtained best practices generally suggest small variations on the daily operation of the hydroelectric station. Hence the water discharge running the turbine will have slight hourly fluctuation, increasing the daily benefits. Between December and February, the best practice suggests an almost uniform flow. This is illustrated in Fig. 5 and can be justified by considering that during winter energy production is higher and the tariff scheme smoother.

On the contrary, in the period between April and October, the optimization suggested significant management changes throughout the day. This can be explained by the lower water availability in these months and subsequently SHP’s lower energy production. The limited water resources are not sufficient to fully operate the SHP, and tariff structure plays the major role in SHP’s management. That explains the curves in Fig. 5 clearly corresponding to peak demand, maximizing energy production when energy prices reach their maximum.

The optimization increases the revenues, especially during low-flow months (+43.8 % in May, Table 2). Optimization’s positive impact is significantly smaller in high-flow months, attenuating the overall annual gains.

Table 2 Monthly benefits with/without optimization of the operation
Full size table

Recent advances in optimization toolkit software

There exists a great variety of optimization toolkits and frameworks aimed at supporting end users and researchers alike in conducting optimization experiments and analyzing their results. This variety is attributed to the plethora of available optimization algorithms, as well as to the diversity of the problems to which these algorithms are applied. In particular, different configuration parameters for each of the considered algorithms and a problem-specific scope that reflects lack of conformity in regard to the definition of constraints and goals, both motivate the large number of available optimization toolkits. As far as optimization is concerned, the seminal work presented in Wolpert and Macready (1997) validated that there is no optimal solution for all types of problems and that the performance of each algorithm greatly depends on the considered problem. Accordingly, for an optimization toolkit to be rendered broadly useful, a major functional requirement is its generality in terms of its support for various optimization algorithms. Additionally, other requirements include extensibility in supporting additional and possibly new algorithms and problems, portability in regard to its operational environment and usability in that it should be user-friendly (Houska et al. 2011).

Early approaches in developing solutions to facilitate optimization processes mostly considered high-level programming interfaces (APIs) that would enable users to build their own optimization tools. Accordingly, PSOt (Birge 2003) is a MATLAB-based particle swarm optimization (PSO) toolkit that includes a suite of targeted custom function calls and low-level commands. The lack of flexibility and the narrow focus of this toolkit hinder its wide applicability, nevertheless other similar approaches were also proposed. Indicatively, PSOTS (Rui et al. 2009) is a SciLab toolbox with a Tcl/Tk interface that aims at single- and multi-objective optimization and in particular exploits PSO variants, OPT++ (Meza et al. 2007) is a C++ library for nonlinear optimization, whereas the ParadiseEO (Liefooghe et al. 2007) software framework provides a generic C++ API for multi-objective metaheuristic algorithms with the goal of supporting minimal programming on behalf of the users and a reusable design. Evidently, such approaches require a programming background and significant efforts from the users, while additionally aspects such as extensibility and generality are difficult to achieve using these interfaces.

Accordingly, works such as pyOPT (Perez et al. 2012) (Python-based object-oriented framework for nonlinear constrained optimization) and PISA (Bleuler et al. 2003) (generic interface for search algorithms) identified the previously mentioned limitations and proposed generic interfaces to support optimization processes independently of the underlying platforms and programming languages. Moreover, the considered decoupling of optimization constraints from application-specific formulations compensates to a great extent the lack of extensibility of such approaches. The work by Fang and Horstemeyer (2006) goes one step further in this direction by introducing a generic API for all multi-objective optimization problems with the goal of reducing the need for user programming. While both the object-oriented approach undertaken and the measured performance (less than 8 % additional optimization time compared to hard-coded, optimizer-specific solutions) are promising, this work suffers from the fact that it does not actually achieve its goal and users need to be savvy in programming to perform optimization tasks.

To alleviate such concerns, GUI-based (graphical user interface) solutions were introduced in order to hide and abstract the complexity of optimization algorithms from the users and allow them to focus only on the functional behavior of these algorithms. The first generation of graphical optimization toolkits consisted of algorithm- or problem-specific solutions that lack generic nature. In this line of work, TSPAntSim (Aybars and Dogan 2009) is a web-based optimization software with a narrow focus on using ant colony optimization (ACO) as the optimizer and on solving problems that are based on the traveling salesman problem. While it is modular and extensible due to its object-oriented Java-based implementation, its scope is very limited to be widely used. Another object-oriented toolkit for optimization is iOpt (Voudouris et al. 2001), which separates presentation from program logic by making use of the MVC software pattern to present an interactive GUI that allows constraint-based problem modeling and optimization. Despite the fact that this work lacks experimental evaluation, it was one of the earliest developments of GUI-based optimization toolkits together with commercial systems such as VisualDOC (Balabanov et al. 2002), which provides a wealth of features, e.g., third party tool integration, database backend, variety of supported algorithms (genetic algorithms, evolutionary algorithms, PSO, and gradient-based optimization), albeit at a high cost and requiring excess computational resources. Further GUI-based optimization toolkits include EVA2 (Kronfeld et al. 2010) (Java-based, client–server optimization framework for evolutionary metaheuristic algorithms), OPT4j (Lukasiewycz et al. 2011) (modular metaheuristic optimization framework), I-EMO (Deb and Chaudhuri 2005) (optimization tool for genetic algorithms supporting their parameterization and results’ visualization) and HeuristicLab (Wagner and Affenzeller 2005) (optimizer-independent and highly modular and extensible optimization environment supporting a great variety of evolutionary algorithms, such as ACO, PSO, and simulated annealing), while it should be noted that this listing is not meant to be exhaustive.

Recently, more advanced toolkits emerged that have extended functionality and address the majority of the aforementioned requirements. In this respect, jMetal (Durill and Nebro 2011) is a Java-based multi-objective optimization framework that supports a variety of evolutionary optimization algorithms and its object-oriented design and implementation inherently allow for code reusability, extensibility, and modularity. Moreover, it provides benchmark tests and quality indicators for performance assessment of various algorithms over a series of problems. A drawback of this framework is its lack of interactivity and the minimal support for decision-making, two aspects which are the focus of the toolbox presented in Tan et al. (2001). The latter involves users in the process of deciding on the optimal solution (also seen in the HuGS middleware platform Klau et al. 2002 that supports rapid prototyping of optimization systems) by means of a MATLAB-based GUI, but suffers from poor execution time compared to C/C++-based solutions and currently only supports a small number of evolutionary algorithms with limited extensibility. However, the fact that it allows for interactive, real-time optimization is a highly desirable feature. A similar approach is followed by the \(\mu\) GP evolutionary optimization toolkit (Sanchez et al. 2011) that is built using C++, but provides no GUI to end users and utilizes cumbersome XML configuration files to introduce problem constraints and algorithmic parameters. Its key feature is the fact that it involves end users in the optimization process by allowing them to set real-world constraints and thus guide the process towards the optimal solution. One of the most interesting and comprehensive open-source software environments for optimization is LiGer (Giagkiozis et al. 2013). It comprises a visual programming language that enables users to design their optimization problems as workflows, the execution of which yields the optimal solution. This Qt-based environment is quite extensible and has a focus on evolutionary optimization. Nonetheless, the latest prototype provides limited support for optimizers, namely differential evolution and PSO.

The use of optimization toolkits is prevalent in the clean energy production sector. A neural network toolbox has been developed to model and optimize the process of biogas recovery from landfills (Behera et al. 2015). As far as the energy efficiency field is concerned, simulation–optimization toolkits have been applied to building performance analysis (Nguyen et al. 2014). MATLAB genetic algorithm toolbox has been recently applied on a multi-objective optimization problem, where an advanced steam power plant is optimized in terms both environmental and economic (Lira-Barragán et al. 2015). The same solver was used in a research on industrial waste optimal heat recovery (Gutiérrez-Arriaga et al. 2015). Such toolkits have also been applied in the optimization of off-grid electrification systems. The optimization of PV-diesel hybrid system for rural areas in Africa has been implemented with the use of the MATLAB optimization toolbox (Kusakana 2015).

Having examined the related literature as far as toolkits for optimization are concerned, we identified the major functional requirements to take into account in our proposed toolkit. In this respect, extensibility, generic nature both in terms of optimizers and problem definition, portability, efficiency, and usability are the key elements of our toolkit.

Conclusions

The previous analysis shows HSA toolkit’s successful use in a sustainable energy optimization problem. Its user-friendly environment, along with its performance and the graphical results, make it a valuable tool. Besides, the more complex the problems to be optimized, the more intense the necessity to adopt user-friendly, neat approaches.

Complex optimization applications are common in the energy field. The need for immediate, even real-time solutions is expected to raise as the share of intermittent energy sources grows. Complexity is also expected to increase as societies shift toward smart, interconnected electricity systems. SHPs role in the energy mix is to balance and mitigate the intermittency of PV and wind production. Thus, interventions to upgrade their power capacity (Kougias et al. 2016a), strategies to support their complementarity with other energy systems (Kougias et al. 2016b) and software supporting their flexible operation such as HSA toolkit can be proved necessary.

Future plans on the HSA toolkit include a version ported to Java, in order to benefit from its inherent portability and extensibility. The possibility to include further optimizers (e.g., ant colony and particle swarm) will promote flexibility and allow users to select the optimal one for their needs. This will also offer the ability to compare solutions obtained by different algorithms and locate global optimum solutions.