This paper presents four Q-learning-based algorithms—Centralized Cooperative Q-Learning (CCQL), Independent Cooperative Q-Learning (ICQL), Cooperative Distributed Q-Learning (CDQL), and Independent Distributed Q-Learning (IDQL)—for optimizing power allocation in Free Space Optical (FSO) Non-Orthogonal Multiple Access (NOMA) networks. The study addresses the critical challenge of inter-cell interference (ICI) in multi-cell FSO networks, leveraging single/multi-agent reinforcement learning to enhance spectral efficiency and user fairness. By integrating real-world FSO channel impairments such as turbulence, path loss and geometric loss into the learning process, we evaluate the algorithms in terms of data sum rate, reward accumulation, and average signal-to-interference-plus-noise ratio. Our results show that IDQL delivers the highest spectral efficiency owing to its fully independent decision-making process, albeit with higher computational complexity. CDQL offers a trade-off between performance and overhead, outperforming ICQL and CCQL while maintaining reasonable complexity. ICQL provides moderate improvements over CCQL, which, despite being the least effective in terms of performance, remains a stable and low-complexity solution for constrained environments. These insights underscore the importance of selecting an appropriate learning paradigm based on system requirements. Future work will explore adaptive learning techniques to dynamically adjust cooperation levels, improving convergence efficiency in large-scale networks.

Keywords