Welcome to this special issue of neural computing and applications on extreme learning machine (ELM) and deep learning networks (DLN). Since 1990s, ELM has been becoming a very important learning methodology for neural networks [1,2,3]. ELM has integrated both machine learning and biological learning mechanisms to train neural networks to perform various tasks including pattern classification, decision making and system modelling in science and engineering [4,5,6]. In recent years, the biological learning features of ELM have stimulated the researcher and engineers to combine ELM with many other learning structures such as DLN and Bayseian networks to perform complex big data processing in many areas [7,8,9,10]. Viewing the rapid development of ELM theory and applications, we planned to organize this special issue a year ago for the readers and neural computing society to report the new ideas and innovations in both ELM and DLN areas. After rigorously reviewing all of received 60 papers on the basis of innovativeness and relevance for all NCA readers, we finally selected 21 high-quality papers for this special issue. The following is the brief introduction of these articles in this special issue.

In ‘‘A new intelligent pattern classifier based on deep-thinking’’, the authors propose a brain-like intelligent pattern classifier, aiming at using the human being’s thinking logics and experience to design the pattern classifiers and avoid the matrix inverse computation in conventional classifier designs. It is seen that the proposed classifier has no parameters to be determined via mathematical optimization. Instead, it is designed by using the correlation principles to construct the pattern clusters at first. The middle-level feature vectors can then be extracted from the statistical information of the correlation information between the input data vectors and the ones stored in each pattern cluster.

For accurate classification purpose, the advanced feature vectors are generated with the moments’ information of the middle-level feature vectors. After that, Bayesian inference is implemented for decision making from the weighted sum of the advanced feature components. In addition, a real-time fine-tuning loop (layer) is designed to adaptively “widen” the border of each pattern clustering region such that the input data vectors can be directly classified once they are located in one of the clustering regions. An experiment for the classification of the handwritten digit images from the MNIST database is performed to show the excellent performance and effectiveness of the proposed brain-like pattern classifier.

In ‘‘Inverse partitioned matrix-based semi-random incremental ELM for regression’’, the authors discuss a modified incremental ELM algorithm for improving the efficiency of the original incremental ELM for pattern classification and system modelling. It is seen that the parameters of even hidden nodes are calculated by fitting the residual error in previous phase and then all existing output weights are recursively updated based on inverse partitioned matrix. The proposed algorithm can further reduce the inefficient hidden nodes and obtain a preferable output weight vector which is always the minimum norm in least-squares sense. Theoretical analyses are evaluated by the experimental results with excellent performance in that the fast convergence rate, generalization capability and structural complexity can be achieved compared with the existing incremental extreme learning machine algorithms.

In “Gait recognition using multichannel convolution neural networks”, the authors investigate the challenging problem of cross-view gait recognition and propose a new gait recognition scheme by utilizing convolution neural networks (CNN). It is seen that a new gait feature representation with three-tuple gait silhouettes (TTGS) is constructed by using consecutive gait silhouette pictures. A multichannel CNN (MCNN) network for processing sequential images in a parallel way is then developed to perform gait recognition. In the algorithm, each of the image dataset is treated as an input channel with its own convolutional kernel, such a novel design makes the algorithm more efficient and robust against uncertainties. The proposed approach is evaluated extensively based on the CASIA gait dataset B for cross-view gait recognition, and the OU-ISIR large population gait dataset is also used to verify the generalization capability of the proposed algorithm with large-scale data. The experimental results show the excellent recognition performance compared with a few existing methods.

In ‘‘Residual deep PCA (RDPCA)-based feature extraction for hyperspectral image classification’’, the authors present a novel RDPCA feature extraction system to process the data with the limited sample size but a relatively high feature dimension. It is seen that the developed system with multiple layers is able to remove the redundancy as well as improve the discrimination. It is further shown that, within each layer, PCA is utilized for layer-wise feature extraction, and the reconstruction residual is fed into the next layer. The layer-wise features are concatenated to form the final output feature. The experimental results show that the capability of nonlinear feature extraction is enhanced and the performance of the proposed RDPCA is much better that the ones of the existing DPCA and PCA algorithms.

In ‘‘Audio-visual cross-modal material surface retrieval’’, the authors propose an algorithm of audio-visual cross-modal retrieval on material surface, aiming at solving the problem that the text and image cannot provide the true and accurate information of the material. It is seen that the local receptive field-based extreme learning machine (ELM–LRF) is first used to extract sound and image features, the sound and image features are then mapped to the subspace by using canonical correlation analysis (CCA) and retrieved by using Euclidean distance. The experimental results show that the proposed algorithm can perform well for material identification.

In ‘‘Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information”, the authors study an online adaptive optimal control problem for a class of continuous-time Markov jump linear systems (MJLSs) with unknown system dynamics by using a parallel reinforcement learning (RL) algorithm. It is seen that before collecting the system input and output data and learning the systems dynamics, the noise is firstly used as the system control input. A novel parallel RL algorithm is then adopted to compute the corresponding N coupled algebraic Riccati equations (AREs) in a parallel way. Through online reinforcement learning, the optimal PI control policy can be implemented and the convergence of the proposed algorithm is also proved. Finally, the effectiveness of this novel control algorithm with learning mechanism is illustrated by two simulation examples.

In “Extreme learning machine-based FNTSM control strategy for electronic throttle”, the authors propose a novel extreme learning machine-based robust control scheme for automotive electronic throttle systems with uncertain dynamics. It is shown that the well-known extreme learning machine (ELM) is used to estimate the upper bound of the lumped uncertainty while a fast nonsingular terminal sliding mode (FNTSM) feedback controller is designed to achieve global stability and finite-time convergence for the closed-loop system. Although the ELM used in this paper has the same structure as the one in the conventional least square-based ELM used for pattern classifications, i.e. the input weights are randomly chosen, the ELM adopted in the closed-loop control system is designed to achieve global control purpose. The output weights of the ELM will be adaptively adjusted in Lyapunov sense from the perspective of global stability of the closed-loop system, rather than local optimization in conventional ELM. The proposed control can thus not only realize the finite-time error convergence, but also need no prior knowledge of lumped uncertainty. Simulation results are demonstrated to verify the excellent tracking performance of the proposed control in comparison with other existing control methods.

In “Object affordance detection with relationship-aware network”, the authors propose a novel relationship-aware convolutional neural network. It is shown that the symbiotic relationship between multiple affordances and the combinational relationship between the affordance and object are used for the algorithm design, to predict the most probable affordance label for each pixel in the object. It is further shown that, different from the existing CNN-based methods that rely on separate and intermediate object detection step, the proposed network directly produces the pixel-wise affordance maps from an input image in an end-to-end manner. Most importantly, there are three key components in the proposed network: (1) the Coord-ASPP module introducing CoordConv in atrous spatial pyramid pooling (ASPP) to refine the feature maps; (2) the relationship-aware module linking the affordances and corresponding objects to explore the relationships, and (3) OSELM (online sequential extreme learning machine) auxiliary attention module focusing on individual affordances further to assist relationship-aware module. The experimental results with the data in two public datasets show the merits of each module and demonstrate the superiority of the proposed network against the existing ones.

In “Hierarchical attentive Siamese network for real-time visual tracking”, the authors present a novel hierarchical attentive Siamese (HASiam) network for high-performance visual tracking. It is seen that a deeper network combined with a shallow one is able to extract more useful features of the target, the spatial and channel-wise attentions applied to different layers can more accurately capture visual attentions on multi-level semantic abstractions. In addition, a non-local attention module is also adopted on the top layer to force the network to pay more attention to the structural dependency of features at all locations during offline training. The proposed HASiam is trained offline in an end-to-end manner, there is no online updating the network parameters needed during tracking. At the end of the paper, the extensive evaluation results demonstrate the good performance of the proposed algorithm compared with a few existing ones. It is expected that the proposed scheme can be applied to numerous vision applications such as visual surveillance systems, robotics and augmented reality.

In “Novel direct remaining useful life estimation of aero-engines with randomly assigned hidden nodes”, the authors aim to improve data-driven prognostics by presenting a novel approach of directly estimating the remaining useful life (RUL) of aero-engines without requiring setting any failure threshold information or estimating degradation states. Specifically, based on the sensory data, RUL estimations are directly obtained through the universal function approximation capability of the extreme learning machine (ELM) algorithm. To achieve this, the features related with the RUL are first extracted from the sensory data as the inputs of the ELM model. Besides, to optimize the number of observed sensors, three evaluation metrics of correlation, monotonicity and robustness are defined and combined to automatically select the most relevant sensor values for more effective and efficient remaining useful life predictions. The validity and superiority of the proposed approach is evaluated by the widely used turbofan engine datasets from NASA Ames prognostics data repository. The proposed approach shows improved RUL estimation applicability at any time instant of the degradation process without determining the failure thresholds. This also simplifies the RUL estimation procedure. Moreover, the random properties of hidden nodes in the ELM learning mechanisms ensure the simplification and efficiency for real-time implementation. Therefore, the proposed approach suits to real-world applications in which prognostics estimations are required to be fast.

In “A machine-learning-enhanced hierarchical multiscale method for bridging from molecular dynamics to continua”, the authors propose a machine-learning-enhanced approach for material failure/defect classification. It is shown that the molecular dynamics simulations in the molecular model are conducted first to generate a dataset, which represents physical phenomena at the nanoscale. The dataset is then used to train a material failure/defect classification model and stress regression models, respectively. The well-trained models are then implemented to study the mechanical behaviours of materials at the macroscale. Multiscale modelling and simulation of a molecule chain and an aluminium crystalline solid are also presented as the applications of the proposed method. At the end of this paper, a few training methods are compared to show the computational efficiency.

In “An event recommendation model using ELM in event-based social network “, the authors propose a novel event recommendation model based on ELM for solving the social issues about event recommendation in our daily life. It is seen that, in the first step, the features of data, including spatial feature, temporal feature, semantic feature, social feature and historical feature, are extracted from event-based social network (EBSN) platforms. The event recommendation is then transformed into a pattern classification problem, where ELM is used as the classifier for recommendation decision making. The authors have conducted extensive experiments by using the EBSN datasets, and the excellent classification and event recommendation results with the proposed model have been achieved.

In “An experimental evaluation of extreme learning machines on several hardware devices”, the authors demonstrate the performances of their ELM hardware implementations with multi-core CPU, graphics processing unit (GPU) and field programmable gate array (FPGA), respectively. It is shown that all of these ELM hardware processing chips can be used to process the big data classification with powerful matrix multiplication and optimization functions. The experimental results show that the speedup ratios of these optimized ELM acceleration hardware are between 10 and 800, and thus the authors suggest that it is suitable to use GPU to accelerate ELM algorithms for large dataset and FPGA for small dataset.

In “Adaptive neural tracking control for automotive engine idle speed regulation using extreme learning machine”, the authors discussed how to solve low engine speed for fuel saving, minimum emissions and disturbance rejection for preventing engine stall in the automotive engine idle speed control by using the proposed ELM-based adaptive neural control algorithm. It is shown that a new adaptation law is designed to update the weights of ELM in the sense of Lyapunov stability to ensure that all of effects of system uncertainties can be eliminated during the control process. Experimental results indicate that the ELM-based adaptive neural control performs very well compared with the classical proportional–integral–derivative (PID), fuzzy-PID, and backpropagation-neural-network-based controllers under the variation of engine load.

In ‘’ ELM-based driver torque demand prediction and real-time optimal energy management strategy for HEVs”, the authors discuss the coordination between the internal combustion engine and the electric machines in road vehicles and present an EML-based predictive optimal energy management strategy. It is seen that ELM algorithm in this research is exploited to perform the driver torque demand prediction for realizing the receding horizon optimization. With the data from the industrial used traffic-in-the-loop (TILP) powertrain simulation platform, both one-step ahead prediction and multi-step ahead predictions are implemented. The prediction results show the excellent performance compared with a few existing prediction methods.

In “Energy management strategy for electric vehicles based on deep Q-learning using Bayesian optimization”, the authors develop a deep Q-learning (DQL)-based energy management strategy (EMS) for electric vehicles (EV). Firstly, the energy management problem is reformulated to satisfy the condition of employing DQL by considering the dynamics of the system. Then, for achieving the minimum of electricity consumption and the maximum of the battery lifetime, the DQL-based EMS is designed to properly split the power demand into two parts, supplied by the battery and super-capacitor, respectively. In addition, a hyper-parameter tuning method and Bayesian optimization (BO) are introduced to optimize the hyper-parameter configuration for the DQL-based EMS. Simulations are carried out with both training and testing datasets to validate the optimality and the adaptability of the DQL-based EMS.

In “Fast nonsingular terminal sliding mode control for permanent magnet Linear motor via ELM”, the authors propose a novel fast nonsingular terminal sliding mode (FNTSM) control strategy using ELM for the control of permanent magnet linear motor (PMLM) systems. It is shown that the developed FNTSM controller is composed of an equivalent control via ELM technique, a compensation control and a reaching control. Distinguished from the traditional ELM for pattern classification, output weights of the proposed ELM are adaptively adjusted by the adaptive law in Lyapunov sense from the global stability point of view, such that the equivalent control of the proposed controller can be flexibly estimated via ELM. Not only can the strong robustness and the faster convergence rate of the closed-loop control be guaranteed, but also the dependence of system dynamics can be further alleviated in the controller design due to the implementation of the ELM. Comparative simulation results are given to validate the robust control performance of the developed controller for both step tracking and sinusoidal tracking purposes.

In “Robust surface reconstruction from highly noisy point clouds using distributed elastic networks”, the author proposes a novel distributed elastic random weight network (DERWN) to implement the robust surface reconstruction from highly noisy point clouds sampled from real surface. The designed elastic regularization with L1 and L2 penalty items makes the network more resilient to noise and effectively capture the intrinsic shapes of surfaces. Especially, the sparsity constraints of output weight vectors and threshold-based nodes removal are very helpful for both determining the appropriate number of hidden nodes and optimizing the distribution of hidden nodes of networks. The distributed optimization in DERWN on the basis of alternating direction method of multipliers (ADMM) solves the problem that traditional RWN learning algorithm suffers from the limitation of memory with large-scale data. Most importantly, the proposed DERWN is able to derive the design solution of global problem by solving local ones. Experimental results show that the proposed DERWN algorithm can robustly reconstruct the unknown surface in case of highly noisy data with satisfying accuracy and smoothness.

In “Hyperspectral image super-resolution using recursive densely convolutional neural network with spatial constraint strategy”, the authors propose a compact deep network for HSI super-resolution (SR) by fusing the idea of recursion, dense connection, and spatial constraint (SCT) strategy. We name this method as recursive densely convolutional neural network with a spatial constraint strategy (SCT-RDCNN). The proposed method uses a novel designed recursive densely convolutional neural network (RDCNN) to learning the mapping relation from the low-resolution (LR) HSI to the high-resolution (HR) HSI, then adopts the SCT to improve the determined HR HSI. Compared with some existing deep-network-based HSI SR methods, the proposed method can use much less parameters (weight and bias) to attain or exceed the performance of methods with similar convolution layers because of the recursive structure and dense connection. It is significant and meaningful for the practical applications of the network in HSI SR due to the limitations of hardware devices. Some experiments on three HIS databases illustrate that our proposed SCT-RDCNN method outperforms several state-of-the-art HSI SR methods.

In “A multi-target corner pooling-based neural network for vehicle detection”, the authors propose a novel convolutional neural network based on multi-target corner pooling layers. It is seen that the proposed network is able to extract both local and global information of the vehicles in the images simultaneously. Instead of using the max pooling layer, the proposed multi-target corner pooling (MTCP) layer can be used to generate the vehicles’ corners feature. And in order to complete the blocked corners that cannot be generated by MTCP, a novel matching corners method (MCM) is adopted in the network, and thus, the proposed network can detect blocked vehicles accurately. Experiments demonstrate that the proposed network achieves an AP of 43.5% on MS COCO dataset and a precision of 93.6% on traffic videos, which outperforms the several existing detecting algorithms.

In “Extreme learning machine-based super-twisting repetitive control for aperiodic disturbance, parameter uncertainty, friction, and backlash compensations of a brushless DC servo motor”, the authors present an extreme learning machine-based super-twisting repetitive control (ELMSTRC) to improve the tracking accuracy of periodic signal with less chattering. The proposed algorithm is robust against the plant uncertainties caused by mass and viscous friction variations. Moreover, it compensates the nonlinear friction, and the backlash by using extreme learning machine-based super-twisting algorithm. Firstly, a repetitive control (RC) is designed to track the periodic reference and compensate the viscous friction. Then, a stable extreme learning machine-based super-twisting control (ELMSTC) is constructed to compensate the aperiodic disturbance, nonlinear friction, backlash and plant uncertainties. The stability of ELMSTRC system is analysed based on Lyapunov stability criterions. The proposed algorithm is verified on a brushless DC servo motor with various loading, backlash and friction conditions. The simulation and experimental comparisons highlight the advantages of the proposed algorithm.

Overall, we hope that you enjoy this special issue and find all of these papers to be influential to your thoughts on neural networks and learning systems. The guest editors wish to thank Editor-in-Chief Professor John MacIntyre for providing the opportunity to organize this special issue. We would also like to thank all of authors, reviewers and Springer Journals Production editors for their efforts in making the special issues possible within the short time.