Interference Mitigation for Millimeter Wave Communications in 5G Networks Using Enhanced Q-Learning
Main Article Content
Abstract
Since the coverage of millimeter waves (mmWave) is limited due to high path loss and blockage, it is deployed in small cells. This dense deployment of base stations and access points resulted in significant interference. Therefore, interference mitigation is the main challenge in designing the new millimeter-wave communication technologies in the existing 5G system. Therefore, in this paper, a two-tier Heterogeneous Cloud Radio Access Network model is presented, which performs a technique inspired by soft frequency reuse (SFR) to mitigate interference. The cellular service region is divided into two sub-regions, center region is served by conventional macro base stations, which operate in the sub-6 GHz frequency band, while the edge area is served by Remote Radio Heads (RRHs), which operate in the millimeter-wave frequency band to avoid interference between tiers. User-RRH associations are introduced to mitigate interference between small RRHs and maximize network throughput using an Online Multi-Agent Q-Learning (MAQL). The proposed MAQL solution, based on the least path loss as a basic criterion for User-RRH association, outperforms in average network throughput per user a previous study based on average SINR as a basic criterion for association for two types of RRHs deployment scenarios in the heterogeneous network approximately by 66.4% and 21%, respectively, at the lowest number of users. The difference gradually decreases with the increasing user numbers until it reaches 8.7% and 9.8%, respectively. Even though the gap between throughput performance narrows as user density increases, the proposed method consistently outperforms the alternative strategy, indicating its ability to adapt and manage network resources more effectively even under higher traffic loads.
Article Details
Section
How to Cite
References
Abdulrezzak, S. and Sabir, F., 2023. An empirical investigation on snort nids versus supervised machine learning classifiers. Journal of Engineering, 29, pp.164–178. https://doi.org/10.31026/j.eng.2023.02.11.
Akdeniz, M.R., Liu, Y., Samimi, M.K., Sun, S., Rangan, S., Rappaport, T.S. and Erkip, E., 2014. Millimeter wave channel modeling and cellular capacity evaluation. IEEE journal on selected areas in communications, 32(6), pp.1164-1179. https://doi.org/10.1109/JSAC.2014.2328154.
Al-Araji, A. and Al-Zangana, S., 2019. Design of new hybrid neural structure for modeling and controlling nonlinear systems. Journal of Engineering, 25, pp.116–135. https://doi.org/10.31026/j.eng.2019.02.08.
Andrews, J.G., Bai, T., Kulkarni, M.N., Alkhateeb, A., Gupta, A.K. and Heath, R.W., 2016. Modeling and analyzing millimeter wave cellular systems. IEEE Transactions on Communications, 65(1), pp.403-430. https://doi.org/10.1109/TCOMM.2016.2618794.
Banar, M., Mohammadi, A. and Kazemi, M., 2022. Characterization of mmwave full-duplex cloud-radio access network (C-RAN) with RRH selection for 5G and beyond. Physical Communication, 52, p.101693. https://doi.org/10.1016/J.PHYCOM.2022.101693.
Checko, A., Christiansen, H., Yan, Y., Scolari, L., Kardaras, G., Berger, M. and Dittmann, L., 2015. Cloud RAN for mobile networks—A technology overview. communications surveys and tutorials, IEEE, 17, pp.405–426. https://doi.org/10.1109/COMST.2014.2355255.
Cheng, S.-H., Liu, J.-L. and Wang, L.-C., 2021. Controlling interference structure and transmit power of aerial small cells by hybrid affinity propagation clustering and reinforcement learning. IEEE Open Journal of Vehicular Technology, 2, pp.412–418. https://doi.org/10.1109/OJVT.2021.3112468.
Dehos, C., González, J.L., De Domenico, A., Kténas, D. and Dussopt, L., 2014. Millimeter-wave access and backhauling: The solution to the exponential data traffic increase in 5G mobile communications systems? IEEE Communications Magazine, 52(9), pp.88–95. https://doi.org/10.1109/MCOM.2014.6894457.
Elsayed, M., Shimotakahara, K. and Erol-Kantarci, M., 2020. Machine learning-based inter-beam inter-cell interference mitigation in mmwave. In: ICC 2020 - 2020 IEEE International Conference on Communications (ICC). pp.1–6. https://doi.org/10.1109/ICC40277.2020.9148711.
Fakhri, Z.H., Sabir, F. and Al-Raweshidy, H.S., 2019. An interference mitigation scheme for millimetre wave heterogeneous cloud radio access network with dynamic RRH clustering. In: 2019 International Symposium on Networks, Computers and Communications (ISNCC). pp.1–8. https://doi.org/10.1109/ISNCC.2019.8909135
Fang, S., Chen, G., Xu, X., Han, S. and Tang, J., 2021. Millimeter-wave coordinated beamforming enabled cooperative network: A stochastic geometry approach. IEEE Transactions on Communications, 69(2), pp.1068–1079. https://doi.org/10.1109/TCOMM.2020.3035387.
Haidine, A., Salmam, F.Z., Aqqal, A. and Dahbi, A., 2021. Artificial intelligence and machine learning in 5g and beyond: A survey and perspectives. In: A. Haidine, ed. Moving Broadband Mobile Communications Forward. [online] Rijeka: IntechOpen. https://doi.org/10.5772/intechopen.98517.
Hajisami, A. and Pompili, D., 2018. Joint virtual edge-clustering and spectrum allocation scheme for uplink interference mitigation in C-RAN. Ad Hoc Networks, 72. https://doi.org/10.1016/j.adhoc.2018.01.010.
Hassan, N. and Fernando, X., 2020. An optimum user association algorithm in heterogeneous 5G networks using standard deviation of the load. Electronics, [online] 9(9). https://doi.org/10.3390/electronics9091495.
Jang, B., Kim, M., Harerimana, G. and Kim, J., 2019. Q-Learning algorithms: A comprehensive classification and applications. IEEE Access, P.1. https://doi.org/10.1109/ACCESS.2019.2941229.
Kai, C., Yi, Y., Peng, M. and Huang, W., 2021. An amplify-and-forward full-duplex cooperative relay scheme for low-latency downlink transmission in CRAN. IEEE Communications Letters, 25(4), pp.1259–1263. https://doi.org/10.1109/LCOMM.2020.3047628.
Kareem Noor, M. and Mosa Omran, B., 2018. BER performance for joint transmission CoMP with SFBC algorithm. International Journal of Innovations in Engineering and Technology, 11(1), P.5. https://doi.org/10.21172/ijiet.111.09.
Khan, M., Fakhri, Z.H. and Al-Raweshidy, H.S., 2018. Semistatic cell differentiation and integration with dynamic BBU-RRH mapping in cloud radio access network. IEEE Transactions on Network and Service Management, 15(1), pp.289–303. https://doi.org/10.1109/TNSM.2017.2771622.
Kolawole, O.Y., Vuppala, S. and Ratnarajah, T., 2018. Multiuser millimeter wave cloud radio access networks with hybrid precoding. IEEE Systems Journal, 12(4), pp.3661–3672. https://doi.org/10.1109/JSYST.2017.2713463.
Kose, A., Lee, H., Foh, C.H. and Shojafar, M., 2024. Multi-agent context learning strategy for interference-aware beam allocation in mmwave vehicular communications. IEEE Transactions on Intelligent Transportation Systems, pp.1–17. https://doi.org/10.1109/TITS.2024.3351488.
Lee, N., Morales-Jimenez, D., Lozano, A. and Heath, R.W., 2015. Spectral efficiency of dynamic coordinated beamforming: A stochastic geometry approach. IEEE Transactions on Wireless Communications, 14(1), pp.230–241. https://doi.org/10.1109/TWC.2014.2337305.
Luo, S., Yang, P., Che, Y.L. and Wu, K., 2020. Space-domain index modulation for mmwave cloud radio access networks. IEEE Transactions on Vehicular Technology, 69(6), pp.6215–6229. https://doi.org/10.1109/TVT.2020.2982700.
Mohammed Sara and Almamori Aqiel, 2024. Cell-free massive MIMO energy efficiency improvement by access points iterative selection. Journal of Engineering, [online] 30(03), pp.129–142. https://doi.org/10.31026/j.eng.2024.03.09.
Mohammed, S. and Hussein, M., 2022. Performance analysis of different machine learning models for intrusion detection systems. Journal of Engineering, 28, pp.61–91. https://doi.org/10.31026/j.eng.2022.05.05.
Nguyen, C., Huynh, N., Chu, N., Saputra, Y., Dinh Thai, H., Nguyen, D., Pham, V., Niyato, T., Dutkiewicz, E. and Hwang, won-J., 2021. Transfer learning for future wireless networks: A comprehensive survey. https://doi.org/10.13140/RG.2.2.10691.53281.
Obi, L., Nche, C., DEUSSOM, E. and Bety, E., 2023. Review of 5G C-RAN resource allocation. EAI Endorsed Transactions on Cognitive Communications, 7, pp.1–28. https://doi.org/10.4108/eetmca.v7i4.3263.
Pompili, D., Hajisami, A. and Viswanathan, H., 2015. Dynamic provisioning and allocation in cloud radio access networks (C-RANs). Ad Hoc Networks, 30. https://doi.org/10.1016/j.adhoc.2015.02.006.
Rodoshi, R.T., Kim, T. and Choi, W., 2020. Resource management in cloud radio access network: Conventional and new approaches. Sensors, [online] 20(9). https://doi.org/10.3390/s20092708.
Simon, M. and Alouini, M., 2005. Digital Communication over Fading Channels. John Wiley & Sons, Inc. https://doi.org/10.1002/0471715220.
Suresh, K., Alqahtani, A., Rajasekaran, T., Kumar, M.S., Ranjith, V., Kannadasan, R., Alqahtani, N. and Khan, A.A., 2022. Enhanced metaheuristic algorithm-based load balancing in a 5G cloud radio access network. Electronics, [online] 11(21). https://doi.org/10.3390/electronics11213611.
Taleb, H., Helou, M. El, Lahoud, S., Khawam, K. and Martin, S., 2018. An efficient heuristic for joint user association and RRH clustering in cloud radio access networks. In: 2018 25th International Conference on Telecommunications (ICT). pp.8–14. https://doi.org/10.1109/ICT.2018.8464852.
Taleb, H., Khawam, K., Lahoud, S., Helou, M. El and Martin, S., 2020. A fully distributed approach for joint user association and RRH clustering in cloud radio access networks. Computer Networks, 182, p.107445. https://doi.org/10.1016/J.COMNET.2020.107445.
Trabelsi, N., Chaari Fourati, L. and Chen, C.S., 2024. Interference management in 5G and beyond networks: A comprehensive survey. Computer Networks, https://doi.org/10.1016/j.comnet.2023.110159.
TSGR, 2022. TR 138 901 - V17.0.0 - 5G; Study on channel model for frequencies from 0.5 to 100 GHz (3GPP TR 38.901 version 17.0.0 Release 17). [online] Available at: https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx.
Wang, X., Turgut, E. and Gursoy, M.C., 2019. Coverage in downlink heterogeneous mmwave cellular networks with user-centric small cell deployment. IEEE Transactions on Vehicular Technology, 68(4), pp.3513–3533. https://doi.org/10.1109/TVT.2019.2895816.
Watkins, C. and Dayan, P., 1992. Q-Learning. Machine Learning, 8, pp.279–292. https://doi.org/10.1007/BF00992698.
yağcıoğlu, M., 2022. Dynamic resource allocation and interference coordination for millimeter wave communications in dense urban environment. Transactions on Emerging Telecommunications Technologies, https://doi.org/10.1002/ett.4442.
Zhu, Q., Wang, C.-X., Hua, B., Kai, M., Jiang, S. and Yao, M., 2021. 3GPP TR 38.901 channel model. pp.1–35. https://doi.org/10.1002/9781119471509.w5gref048.