SMARTWIN: smart reinforcement learning based digital twin for resource optimization in O-RAN

The exponential increase of heterogeneous devices and vertical applications in 5G and Beyond 5G (B5G) networks has catalysed a paradigm shift in cellular network design, fostering a transition towards disaggregated, fully virtualized, and programmable architectures. To meet these demands, the Open Radio Access Network (O-RAN) architecture standardized by the O-RAN Alliance enables hardware independence, while the use of Digital Twins (DTs) for network emulation and validation is becoming increasingly popular. Although O-RAN introduces new technologies and opportunities, advances in Machine Learning (ML) based network automation remain limited, mainly because of insufficient large-scale datasets and experimental testing environments. To address the challenges of accurate network modelling and efficient resource management in O-RAN, this paper introduces SMARTWIN, a Smart Reinforcement Learning based Digital Twin framework. SMARTWIN enables precise network modelling and intelligent resource allocation and scheduling, with the objective of optimizing the Key Performance Indicators (KPIs) of eMBB, mMTC, and URLLC network slices. It also implements a Conservative Q-Learning (CQL) algorithm to learn from data generated by the DT, where the performance comparison with the baseline Implicit Q-Learning (IQL) algorithm shows an improvement of approximately 41% across all slices. This indicates that the proposed SMARTWIN framework can generalize well beyond the behaviour policy, enabling more efficient and intelligent resource allocation decisions within the DT.