Interference Mitigation based on Joint Optimization of NTBS 3D Positions and RIS Reflection in Downlink NOMA HetNets

The next generation of wireless networks is poised to deliver dependable, extensive, and extremely low-latency mobile broadband communications across various industries. In this regard, reconfigurable intelligent surface (RIS)-aided non-terrestrial base station (NTBS) is crucial for developing three-dimensional (3D) networks that aim to integrate the terrestrial and non-terrestrial infrastructures. The integration of RISs with NTBSs has resulted in additional capabilities to attain faster and more flexible mobile coverage. In this paper, we employ a novel RIS-aided NTBSs with non-orthogonal multiple access in downlink heterogeneous networks (HetNets), in which the RIS is utilized to improve the transmission of signals from multiple NTBSs to ground users (GUs). We aim to maximize the system sum rate (SSR) and energy efficiency (EE) by mitigating inter-cluster and intra-cluster interference. The formulated optimization problem is non-convex due to the joint optimization of NTBS 3D positions, RIS reflection angles, RIS reflection coefficients, and successive interference cancellation among GUs. To address this problem, a modified gray wolf optimization -based meta-heuristic algorithm is proposed. In particular, the original optimization problem is divided into three sub-problems (i.e., RIS reflection angles, RIS reflection coefficients, and NTBS 3D position), which are then addressed alternately using the proposed optimization technique. The novelty of our proposed work is presented by reformulating the distance equations to minimize the distance between the wolves and their prey and enhance the search process’s precision. The simulation results stated that the suggested algorithm outperforms the traditional schemes in terms of both SSR and EE. Furthermore, the results show that incorporating RIS into multi-NTBS HetNets effectively boosts overall performance by improving the channel quality between NTBSs and their respective GUs while reducing inter-NTBS interference.