Efficient Deep Learning of Kerr Nonlinearity in Fiber-Optic Channels Using a Convolutional Recurrent Neural Network

Abstract

The impairments arising from the Kerr nonlinearity in optical fiber are a major obstacle in fiber-optic transmission systems. To compensate for these impairments at the receiver, the complexity of the digital signal processing algorithms must be reduced. Deep learning-based equalizers have shown to be promising in this area. However, their efficient implementation in practical systems is still an open problem. In this paper, we propose a low-complexity convolutional recurrent neural network (CNN+RNN) for deep learning of Kerr nonlinearity effects in long-haul optical fiber channels governed by the nonlinear Schrödinger equation. This approach reduces computational complexity by balancing the computational load via capturing short-range temporal features using multi-channel strided convolution layers with ReLU activation, and the long-range temporal features using a unidirectional vanilla recurrent layer. We demonstrate that for a 16-QAM 64 GBd optical transmission system over 1120 km of standard single-mode fiber with 14 spans, the proposed model approaches the performance of digital backpropagation and achieves superior or comparable performance to recently-proposed MLP, CNN+MLP, bi-RNN, bi-GRU, bi-LSTM, and CNN+bi-LSTM-based equalizers in the literature, with substantially fewer floating-point operations (FLOPs) than these models.