Digital back-propagation optimization for high-baudrate single-channel optical fiber transmissions

For future high-speed transmission systems, increasing the symbol rate and modulation format orders have been considered as a main direction to reduce the cost per bit. On the other hand, increasing the symbol rate and modulation format order leads to increase of fiber nonlinear impairments which limit the system performance and reach. Digital back-propagation (DBP) has been considered as one of the most effective techniques for combating the fiber nonlinearity. However, conventional DBP scheme employing standard split-step Fourier method (S-SSFM) requires high hardware computational complexity which is difficult to meet in practice. In this paper, we study the effectiveness of the DBP technique for high-baudrate single-channel transmission systems and propose an advanced DBP technique based on the optimization of a general case of logarithmic step-size and Kerr nonlinearity coefficient. We show that the proposed method improves performance significantly in comparison with S-SSFM and the conventional natural logarithmic step-size technique. In particular, for 40 Gbaud DP-16QAM transmission over 2400 km, the proposed method with 6 steps/span shows 1.3 dB performance advantage over conventional logarithmic step-size with the same complexity. For large data rate (i.e. 160 Gbaud) transmission system, an improvement by 1 dB can be achieved by the proposed technique with 6 steps/span with respect to logarithmic step-size over 2400 km.