Research Article: Master-Leader-Slave Cuckoo Search with Parameter Control for ANN Optimization and Its Real-World Application to Water Quality Prediction

Date Published: January 26, 2017

Publisher: Public Library of Science

Author(s): Najmeh Sadat Jaddi, Salwani Abdullah, Marlinda Abdul Malek, Wen-Bo Du.


Artificial neural networks (ANNs) have been employed to solve a broad variety of tasks. The selection of an ANN model with appropriate weights is important in achieving accurate results. This paper presents an optimization strategy for ANN model selection based on the cuckoo search (CS) algorithm, which is rooted in the obligate brood parasitic actions of some cuckoo species. In order to enhance the convergence ability of basic CS, some modifications are proposed. The fraction Pa of the n nests replaced by new nests is a fixed parameter in basic CS. As the selection of Pa is a challenging issue and has a direct effect on exploration and therefore on convergence ability, in this work the Pa is set to a maximum value at initialization to achieve more exploration in early iterations and it is decreased during the search to achieve more exploitation in later iterations until it reaches the minimum value in the final iteration. In addition, a novel master-leader-slave multi-population strategy is used where the slaves employ the best fitness function among all slaves, which is selected by the leader under a certain condition. This fitness function is used for subsequent Lévy flights. In each iteration a copy of the best solution of each slave is migrated to the master and then the best solution is found by the master. The method is tested on benchmark classification and time series prediction problems and the statistical analysis proves the ability of the method. This method is also applied to a real-world water quality prediction problem with promising results.

Partial Text

Computational intelligence is defined as a set of nature-inspired computational approaches to deal with complex real-world problems. This intelligence is directly linked to computing concepts such as fuzzy logic, decision making, artificial neural networks (ANNs), and metaheuristic algorithms as optimization techniques. Artificial neural networks are a family of learning models that are inspired by biological neural networks and are employed to estimate functions that are generally unknown. A number of researchers have used optimization algorithms to train neural network models [1–8]. The multi-layer self-organizing ANN has been studied in the literature and metaheuristic algorithms have been used to optimize the structure of the ANN [9–12]. A few methods have been used to attempt to optimize both the weights and the structure of ANNs [8, 13–15]. Other proposals include the use of an adaptive merging and growing algorithm in the design of ANNs [16] and the adoption of a Taguchi-based parameter for the genetic algorithm used in ANN training [17]. [4]used a pruned probabilistic neural network with a genetic algorithm to optimize the structure of an ANN, while [6] applied particle swarm optimization (PSO) to optimize an ANN. Other PSO based approaches can be found in [18, 19]. In another research study [20], the authors used a combination of self-organizing networks and an artificial immune system to minimize the neurons in an ANN.

Some cuckoo species lay their eggs in the nest of another bird species alongside the host bird’s eggs, while others remove the host eggs and then lay their own in the nest to increase the hatching probability of their own eggs [30]. Female parasitic cuckoos are specialized in mimicking the pattern and color of the eggs of a few selected host species. This decreases the likelihood of the eggs being abandoned and so improves the reproductive outcome for the cuckoo. However, if a host bird discovers that the eggs are not its own, then it either throws them away or just abandons its nest and makes a new nest elsewhere. The parasitic cuckoo often selects a nest anywhere the host bird has just laid its eggs. Generally, the cuckoo eggs hatch a little before the host bird’s eggs. When the first cuckoo chick is hatched, its first natural action is to throw out the host eggs by blindly pushing the eggs out of the nest. This behavior has the effect of increasing the cuckoo chicks’ share of the food provided by the host bird. A cuckoo chick can also mimic the call of the host chicks to increase its chances of being fed.

The first version of the CS algorithm was introduced by [30]. The authors made three idealized assumptions to ensure the simplicity of the CS algorithm. In this simplified form:

We propose two modifications of CS to improve the performance of the basic CS. The first modification aims to enhance the overall performance of CS by controlling the Pa parameter. Controlling this parameter improves the balance between exploration and exploitation of the search space and can therefore increase the likelihood of fast convergence to the global optimum. The second modification involves maintaining the diversity of the solutions and directing the search process toward the best solution by using a novel master-leader-slave multi-population strategy. The details of these modifications are discussed in the following subsections.

For the ANN used in this present study, two hidden layers with two nodes for each hidden layer were selected as this format is commonly used and is the most accurate [8, 29]. The activation function used in this experiment was the hyperbolic tangent as it has better presentation [39] than other activation functions. A one-dimensional vector was used for the solution representation, where the weights and biases of the ANN are located in each cell of this vector. The length of the vector is equivalent to the number of weights plus the number of biases of the ANN.

This paper examined the capability of the cuckoo search algorithm and its modifications to contribute to a more accurate ANN model. To attain this important aim, first, the basic cuckoo search algorithm was applied to optimize the ANN model and then two modifications (PaCtrl-CS and Multipop-PaCtrl-CS) of the cuckoo search were proposed. These modifications were designed to improve the exploration and exploitation of the algorithm and its ability to achieve better convergence. Control of the Pa parameter in PaCtrl-CS was achieved by setting this parameter to a maximum value in the initial stage to gain more exploration and by decreasing this parameter during the search process to gain more exploitation until the algorithm reached final convergence with the minimum value of Pa. Furthermore, a master-leader-slave multi-population strategy was embedded in Multipop-PaCtrl-CS to improve the convergence ability of the algorithm. In this strategy, the slaves with aid of leader guidance provided good exploration and the master had the role of providing the algorithm with more exploitation. Based on extensive evaluations it is concluded that the Multipop-PaCtrl-CS algorithm has the ability to outperform other recent algorithms in the literature in five out of six classification problems. The algorithm also demonstrated better performance on two tested time series prediction problems. We believe that the superiority of the results is due to the fine balancing between exploration and exploitation in Multipop-PaCtrl-CS provided by Pa control and the master-leader-slave multi-population. Finally, the proposed methods were applied to real-world data for water quality prediction. The promising results for both benchmark and real-world data motivate us to improve this method in future work.




0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments