End-to-End Learning of Constellation Shaping for Optical Fiber Communication Systems

End-to-end learning based on autoencoder can realize robust constellation shaping for optical fiber communications. The existing schemes use the symbol-wise autoencoder (SAE) or bit-wise autoencoder (BAE) to realize the constellation shaping. The SAE mainly focus on the performance of mutual information (MI), this neglects the decoding loss so that the generalized mutual information (GMI) or the post forward error correction (FEC) bit error rate (BER) has almost no performance gain in bit-wise metric systems. In this paper, we propose a probabilistic shaping (PS) based on BAE with a modified loss function, where the mean square error and source entropy are used to construct the loss function. We compare the GMI and post-FEC performance of the PS and also geometric shaping (GS) based on SAE or BAE by numerical simulations and experiments. In simulations, we transmit 64-QAM signal with GS or PS over 100-km SSFM. The simulation results show that the GS or PS based on BAE can achieve 0.13-bits/sym or beyond 0.2-bits/sym GMI gain. In experiment, the GS based on BAE obtains 0.11-bits/sym GMI gain and 0.7-dB launch optical power gain after belief propagation decoding. The PS with source entropy of 5.5-bits/sym and 5.2-bits/sym outperforms uniform 64-QAM by 0.25-bits/sym and 0.3-bits/sym, respectively.