Mobility-Aware and Privacy-Preserving Federated Reinforcement Learning with Multi-Paradigm Machine Learning for Edge Intelligence in 5G/6G Networks
Main Article Content
Abstract
The rise of 5G and 6G networks, along with the rapid growth of edge computing, is creating a strong need for smarter and more privacy-aware ways to handle task offloading as users move across the network. Many current methods still treat mobility prediction, federated learning (FL), and differential privacy (DP) as separate pieces, which often leads to avoidable delays, higher energy use, and weaker data protection. This paper introduces Mobility-Aware Federated Reinforcement Learning (MA-FRL), a framework designed to bring these components together. It integrates deep reinforcement learning (DRL) with supervised and unsupervised ML techniques to enhance edge intelligence, mobility prediction using Markov chains, and Gaussian Differential Privacy (DP) to make better offloading decisions across multi-tier edge environments. MA-FRL uses a federated deep Q-network (DQN), where each edge node trains locally on mobility-aware data and adds DP noise before contributing to the global model. It utilizes NS-3 and م, in addition to real datasets like CRAWDAD, GeoLife, and SPEC power; the framework is among the first to achieve 32% lower latency, 27% energy savings, and strong privacy protection (ε < 1.0). Pareto analysis shows a balance between performance goals and topology-aware tuning, improving results in urban, rural, and vehicular settings. MA-FRL also aligns with the General Data Protection Regulation (GDPR). Future work will explore Long Short-Term Memory (LSTM) and Spatio-Temporal Graph Neural Networks (ST-GNN) mobility models and hardware-in-the-loop testing.
Downloads
Article Details
Section
How to Cite
References
Alsahfi, T., Badshah, A., Alsini, R., Alallah, F. S., Bedewi, W., and Daud, A., 2025. Proximal policy optimization for vehicular big data offloading across edge, regional, and cloud layers. Journal of Grid Computing, 23(28). https://doi.org/10.1007/s10723-025-00328-x.
Amiri, M.M., Gündüz, D., Kulkarni, S.R. and Poor, H.V., 2021. Convergence of update-aware device scheduling for federated learning at the wireless edge. IEEE Transactions on Wireless Communications, 20(6), pp. 3643-3658. https://doi.org/10.1109/TWC.2021.3052681
Bai, Y., Zhao, H., Zhang, X., Chang, Z., Jäntti, R. and Yang, K., 2023. Toward autonomous multi-UAV wireless network: A survey of reinforcement learning-based approaches. IEEE Communications Surveys and Tutorials, 25(4), pp. 3038-3067. https://doi.org/10.1109/COMST.2023.3323344
Betalo, M.L., Ullah, I., Tesema, F.B., Wu, Z., Li, J. and Bai, X., 2025. Generative AI-Driven Multi-Agent DRL for task allocation in UAV-Assisted EMPD within 6G-Enabled SAGIN networks. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2025.3579780
Bi, R., Guo, D., Zhang, Y., Huang, R., Lin, L. and Xiong, J., 2023. Outsourced and privacy-preserving collaborative k-prototype clustering for mixed data via additive secret sharing. IEEE Internet of Things Journal, 10(18), pp. 15810-15821. https://doi.org/10.1109/JIOT.2023.3266028
Cao, K., Chen, M., Karnouskos, S. and Hu, S., 2024. Reliability-aware personalized deployment of approximate computation IoT applications in serverless mobile edge computing. IEEE Transactions on computer-aided design of integrated circuits and systems, 44(2), pp. 430-443. https://doi.org/10.1109/TCAD.2024.3437344.
Chafii, M., Naoumi, S., Alami, R., Almazrouei, E., Bennis, M. and Debbah, M., 2023. Emergent communication in multi-agent reinforcement learning for future wireless networks. IEEE Internet of Things Magazine, 6(4), pp. 18-24. https://doi.org/10.1109/IOTM.001.2300102
Chen, X., Feng, D., Jiang, W., Luo, Q., Chen, G., and Sun, Y., 2025. QoE-driven multi-task offloading for semantic-aware edge computing systems. IEEE Transactions on Mobile Computing. Advance online publication. https://doi.org/10.1109/TNSE.2026.3662899
Chen, Y., Zhao, L., and Wang, X., 2026. Privacy preserving federated averaging with adaptive communication. ACM Transactions on Intelligent Systems and Technology, 17(3), pp. 45–67. https://doi.org/10.1145/3623456
CRAWDAD., 2022. A Community Resource for Archiving Wireless Data at Dartmouth. https://crawdad.org/~crawdadarchive/all-bydate.html
Dai, Z., Zhang, Y., Zhang, W., Luo, X. and He, Z., 2022. A multi-agent collaborative environment learning method for UAV deployment and resource allocation. IEEE Transactions on Signal and Information Processing over Networks, 8, pp. 120-130. https://doi.org/10.1109/TSIPN.2022.3150911
Deng, M., Liang, W., Qiu, B., and Xie, X., 2025. Federated deep reinforcement learning-based task offloading strategy with game theory in vehicular edge computing. Physical Communication, 65, P. 102802. https://doi.org/10.1016/j.phycom.2025.102802
Dritsas, E., Ramantas, K. and Verikoukis, C.V., 2024, September. A mobility-aware reinforcement learning proactive solution for state data migration in edge computing. In 2024 IEEE 29th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), pp. 1–6. IEEE.
https://doi.org/10.1109/CAMAD62243.2024.10943072
Duan, Q., Huang, J., Hu, S., Deng, R., Lu, Z. and Yu, S., 2023. Combining federated learning and edge computing toward ubiquitous intelligence in 6G network: Challenges, recent advances, and future directions. IEEE Communications Surveys and Tutorials, 25(4), pp. 2892-2950. https://doi.org/10.1109/COMST.2023.3316615
Feng, C., Yang, H.H., Hu, D., Zhao, Z., Quek, T.Q. and Min, G., 2022. Mobility-aware cluster federated learning in hierarchical wireless networks. IEEE Transactions on Wireless Communications, 21(10), pp. 8441-8458. https://dx.doi.org/10.1109/TWC.2022.3166386
Glanois, C., Weng, P. , Zimmer, M., Li, D., Yang, T., Hao, J. and Liu, W., 2024. A survey on interpretable reinforcement learning. Machine Learning, 113(8), pp. 5847-5890. https://doi.org/10.1007/s10994-024-06543-w
Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, J. and Knoll, A., 2024. A review of safe reinforcement learning: Methods, theories, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), pp. 11216-11235. https://doi.org/10.1109/TPAMI.2024.3457538
Guo, Y., Xie, H., Wang, C. and Jia, X., 2021. Enabling privacy-preserving geographic range query in fog-enhanced IoT services. IEEE Transactions on Dependable and Secure Computing, 19(5), pp. 3401-3416. https://doi.org/10.1109/TDSC.2021.3095933
Hamdi, A.M.A., Hussain, F.K. and Hussain, O.K., 2022. Task offloading in vehicular fog computing: State-of-the-art and open issues. Future Generation Computer Systems, 133, pp. 201-212. https://doi.org/10.1016/j.future.2022.03.019
Hasselt, H. V., Hessel, M., and Aslanides, J., 2024. Deep reinforcement learning with double Q learning revisited. Journal of Machine Learning Research, 25(112), pp. 1–35. https://doi.org/10.5555/jmlr.v25.112
Hsieh, L.T., Liu, H., Guo, Y. and Gazda, R., 2023. Deep reinforcement learning-based task assignment for cooperative mobile edge computing. IEEE Transactions on Mobile Computing, 23(4), pp. 3156-3171. https://doi.org/10.1109/TMC.2023.3270242.
Jiang, H., Dai, X., Xiao, Z. and Iyengar, A., 2022. Joint task offloading and resource allocation for energy-constrained mobile edge computing. IEEE Transactions on Mobile Computing, 22(7), pp. 4000-4015. https://doi.org/10.1109/TMC.2022.3150432
Khot, V., Pai, S.S. and Rao, V.R., 2023, September. Deep Reinforcement Learning for Task Offloading in a Multi-Access Edge Computing Environment. In 2023 International Conference on Network, P. 91. https://doi.org/10.1109/NMITCON58196.2023.10275998.
Lekharu, A., Samanta, A., Sur, A. and Patra, M., 2024. Content-aware caching at the mobile edge network using federated learning. IEEE Transactions on Emerging Topics in Computational Intelligence, 9(2), pp. 1166-1176. https://doi.org/10.1109/TETCI.2024.
Levine, S., Kumar, A., and Finn, C., 2025. Reinforcement learning: Foundations, advances, and frontiers. Annual Review of Control, Robotics, and Autonomous Systems, 8(1), pp. 1–35. https://doi.org/10.1146/annurev-cras-080124-010321
Li, B., Ma, S., Deng, R., Choo, K.K.R. and Yang, J., 2022. Federated anomaly detection on system logs for the Internet of Things: A customizable and communication-efficient approach. IEEE Transactions on Network and Service Management, 19(2), pp. 1705-1716. https://doi.org/10.1109/TNSM.2022.3152620
Li, C., Zhang, Y. and Luo, Y., 2022. A federated learning-based edge caching approach for mobile edge computing-enabled intelligent connected vehicles. IEEE Transactions on Intelligent Transportation Systems, 24(3), pp. 3360-3369. https://doi.org/10.1109/TITS.2022.3224395.
Li, T., Zhu, K., Luong, N.C., Niyato, D., Wu, Q., Zhang, Y. and Chen, B., 2022. Applications of multi-agent reinforcement learning in future internet: A comprehensive survey. IEEE Communications Surveys and Tutorials, 24(2), pp. 1240-1279. https://doi.org/10.1109/COMST.2022.3160697
Liang, Z., Wang, B., Gu, Q., Osher, S. and Yao, Y., 2024. Differentially private federated learning with Laplacian smoothing. Applied and Computational Harmonic Analysis, 72, P. 101660. https://doi.org/10.1016/j.acha.2024.101660.
Liu, H., and Sun, F., 2025. Stabilizing deep Q networks with distributional reinforcement learning. IEEE Transactions on Artificial Intelligence, 5(1), pp. 22–34. https://doi.org/10.1109/TAI.2025.1239876
Liu, Y., Xue, E., Chai, W., Liu, Y., and Feng, F., 2026. Offloading and computational resources allocation of tasks with dependency in MEC environment based on deep reinforcement learning. Ad Hoc Networks, 154, P. 104136. https://doi.org/10.1016/j.adhoc.2026.104136
Loutfi, S.I., Shayea, I., Tureli, U., El-Saleh, A.A. and Tashan, W., 2024. An overview of mobility awareness with mobile edge computing over 6G network: Challenges and future research directions. Results in Engineering, 23, P. 102601. https://doi.org/10.1016/j.rineng.2024.102601.
Lu, J., Liu, H., Zhang, Z., Wang, J., Goudos, S.K. and Wan, S., 2021. Toward fairness-aware time-sensitive asynchronous federated learning for critical energy infrastructure. IEEE Transactions on Industrial Informatics, 18(5), pp. 3462-3472. https://dx.doi.org/10.1109/TII.2021.3117861
Ma, C., Li, A., Du, Y., Dong, H. and Yang, Y., 2024. Efficient and scalable reinforcement learning for large-scale network control. Nature Machine Intelligence, 6(9), pp. 1006-1020. https://doi.org/10.1038/s42256-024-00879-7
Min, M., Liu, Z., Duan, J., Zhang, P. and Li, S., 2023. Safe-learning-based location-privacy-preserved task offloading in mobile edge computing. Electronics, 13(1), P. 89. https://doi.org/10.3390/electronics13010089
Mitsis, G., Tsiropoulou, E.E. and Papavassiliou, S., 2022. Price and risk awareness for data offloading decision-making in edge computing systems. IEEE Systems Journal, 16(4), pp. 6546-6557. https://doi.org/10.1109/JSYST.2022.3188997
Musa, S.S., Zennaro, M., Libsie, M. and Pietrosemoli, E., 2022. Mobility-aware proactive edge caching optimization scheme in information-centric IoV networks. Sensors, 22(4), P. 1387. https://doi.org/10.3390/s22041387
Nahar, A., Sikarwar, H., Jain, S. and Das, D., 2023, May. Cachein: A secure distributed multi-layer mobility-assisted edge intelligence-based caching for Internet of vehicles. In 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pp. 437-446. IEEE. https://dx.doi.org/10.1109/CCGrid57682.2023.00048
Qian, Z., Feng, Y., Dai, C., Li, W. and Li, G., 2024. Mobility-aware proactive video caching based on asynchronous federated learning in mobile edge computing systems. Applied Soft Computing, 162, P. 111795. https://doi.org/10.1016/j.asoc.2024.111795
Rui, L., Wang, S., Wang, Z., Xiong, A. and Liu, H., 2021. Dynamic service migration strategy based on mobility prediction in edge computing. International Journal of Distributed Sensor Networks, 17(2), pp. 1–12. SAGE Publications. https://doi.org/10.1177/1550147721993403
Serghiou, D., Khalily, M., Brown, T.W. and Tafazolli, R., 2022. Terahertz channel propagation phenomena, measurement techniques and modeling for 6G wireless communication applications: A survey, open challenges and future research directions. IEEE Communications Surveys and Tutorials, 24(4), pp. 1957-1996.
https://doi.org/10.1109/COMST.2022.3205505
Song, X., Chen, Q., Wang, S., and Song, T., 2025. Cross-domain resources optimization for hybrid edge computing networks: Federated DRL approach. Digital Communications and Networks, 11(6), pp. 1797–1808. https://doi.org/10.1016/j.dcan.2024.03.006.
SPECpower. 2008. Results Search tool. https://www.spec.org/power_ssj2008/results/.
Sutton, R. S., and Barto, A. G., 2018. Reinforcement learning: An introduction (2nd ed.). MIT Press. https://doi.org/10.7551/mitpress/11177.001.0001
Tahmid, T., Gates, M., Luszczek, P. and Schuman, C., 2024. Towards scalable and efficient spiking reinforcement learning for continuous control tasks. In 2024 International Conference on Neuromorphic Systems (ICONS), pp. 331-335, IEEE. https://doi.org/10.1109/ICONS62911.2024.00057
Tang, J., Qin, T., Xiang, Y., Zhou, Z. and Gu, J., 2022. Optimization search strategy for task offloading from collaborative edge computing. IEEE Transactions on Services Computing, 16(3), pp. 2044-2058. https://doi.org/10.1109/TSC.2022.3203700
Tang, J., Wang, S., Yang, S., Xiang, Y. and Zhou, Z., 2024. Federated learning-assisted task offloading based on feature matching and caching in collaborative device-edge-cloud networks. IEEE Transactions on Mobile Computing, 23(12), pp. 12061-12079. https://doi.org/10.1109/TMC.2024.3403851.
Ullah, I., 2025. Digital twin-empowered task offloading in VEC: Attention-augmented DRL for mobility-resilient optimization. Computer Networks, 245, P. 111818. https://doi.org/10.1016/j.comnet.2025.111818
Wang, G., Liang, R., Hu, J., 2021. Resource caching and task migration strategy of small cellular networks under mobile edge computing. IEEE Transactions on Mobile Computing, 20(11), pp. 3137–3151. https://doi.org/10.1109/TMC.2020.2981897
Wang, W., Zhao, Y., Wu, Q., Fan, Q., Zhang, C. and Li, Z., 2022. Asynchronous federated learning-based mobility-aware caching in vehicular edge computing. In 2022 14th International Conference on Wireless Communications and Signal Processing (WCSP), pp. 1-5. https://dx.doi.org/10.1109/WCSP55476.2022.10039430
Wang, Y., Qian, Z., He, L., Yin, R. and Wu, C., 2024. Intelligent online computation offloading for wireless-powered mobile-edge computing. IEEE Internet of Things Journal, 11(17), pp. 28960-28974. https://doi.org/10.1109/JIOT.2024.3406204
Wei, K., Li, J., Ding, M., Ma, C., Yang, H.H., Farokhi, F., Jin, S., Quek, T.Q. and Poor, H.V., 2020. Federated learning with differential privacy: Algorithms and performance analysis. IEEE Transactions on Information Forensics and Security, 15, pp. 3454-3469. https://doi.org/10.1109/TIFS.2020.2988575.
Wu, Q., Zhao, Y., Fan, Q., Fan, P. , Wang, J. and Zhang, C., 2022. Mobility-aware cooperative caching in vehicular edge computing based on asynchronous federated and deep reinforcement learning. IEEE Journal of Selected Topics in Signal Processing, 17(1), pp. 66-81. https://doi.org/10.1109/JSTSP.2022.322127d1.
Wu, Z., Fang, H., Tang, J., and Yang, X., 2025. Lightweight adaptive PPO-AHP enhanced algorithm for task offloading in vehicular edge computing. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 1–9. https://doi.org/10.1109/IJCNN.2025. 9568743
Xiao, Y., Xiao, L., Wan, K., Yang, H., Zhang, Y., Wu, Y. and Zhang, Y., 2022. Reinforcement learning based energy-efficient collaborative inference for mobile edge computing. IEEE Transactions on Communications, 71(2), pp. 864-876. https://doi.org/10.1109/TCOMM.2022.3229033.
Xie, B. and Cui, H., 2025. Deep reinforcement learning-based dynamical task offloading for mobile edge computing: B. Xie, H. Cui. The Journal of Supercomputing, 81(1), P. 35. https://doi.org/10.1007/s11227-024-06603-x.
Xie, B., Sun, Y., Zhou, S., Niu, Z., Xu, Y., Chen, J. and Gunduz, D., 2023, May. MOB-FL: Mobility-aware federated learning for intelligent connected vehicles. In ICC 2023-IEEE International Conference on Communications. pp. 3951-3957. https://doi.org/10.1109/ICC45041.2023.10279773.
Xu, Z., Kairouz, P. , McMahan, H. B., and Ramage, D., 2025. Federated optimization in heterogeneous networks: Advances and open problems. IEEE Transactions on Neural Networks and Learning Systems, 36(2), pp. 345–362. https://doi.org/10.1109/TNNLS.2025.1234567
Yamansavascilar, B., Baktir, A.C., Sonmez, C., Ozgovde, A. and Ersoy, C., 2022. Deepedge: A deep reinforcement learning based task orchestrator for edge computing. IEEE Transactions on Network Science and Engineering, 10(1), pp. 538-552. https://doi.org/10.1109/TNSE.2022.3217311.
Yu, G., He, Y., Wu, J., Chen, Z. and Pan, J., 2023. Mobility-aware proactive edge caching for large files in the internet of vehicles. IEEE Internet of Things Journal, 10(13), pp. 11293-11305. https://dx.doi.org/10.1109/JIOT.2023.3240423
Zhang, H., Ma, T., Chen, G., Ni, Y., Liu, T., and Zhou, H., 2025. Federated learning-driven computation offloading in space-air-ground integrated networks. In Proceedings of the 2025 Seventeenth International Conference on Wireless Communications and Signal Processing (WCSP), pp. 1–6. https://doi.org/10.1109/WCSP68525.2025.1010157
Zhang, J., Cheng, L., Liu, C., Zhao, Z. and Mao, Y., 2023. Cost-aware scheduling systems for real-time workflows in cloud: An approach based on genetic algorithm and deep reinforcement learning. Expert Systems with Applications, 234, P. 120972. https://doi.org/10.1016/j.eswa.2023.120972
Zhang, P. , Gan, P. , Chang, L., Wen, W., Selvi, M. and Kibalya, G., 2022. DPRL: Task offloading strategy based on differential privacy and reinforcement learning in edge computing. IEEE Access, 10, pp. 54002-54011. http://dx.doi.org/10.1109/ACCESS.2022.3175194.
Zhang, P. , Wang, E., Guizani, M., Liu, K., Wang, J., and Tan, L., 2025. Privacy-preserving task offloading in vehicular edge computing using federated multi-agent reinforcement learning. IEEE Transactions on Vehicular Technology, 74(12), pp. 19642–19654. https://doi.org/10.1109/TVT.2025.3588204
Zhang, R., Xia, H., Chen, Z., Kang, Z., Wang, K. and Gao, W., 2024. Computation cost-driven offloading strategy based on reinforcement learning for consumer devices. IEEE Transactions on Consumer Electronics, 70(1), pp. 4120-4131. https://doi.org/10.1109/TCE.2024.3355455
Zhang, Y., and Wang, J., 2026. Advances in deep Q networks for complex decision making. Neural Computation, 38(4), pp. 789–812. https://doi.org/10.1162/neco_a_01567
Zhou, X., Bilal, M., Dou, R., Rodrigues, J.J., Zhao, Q., Dai, J. and Xu, X., 2023. Edge computation offloading with content caching in 6G-enabled IoV. IEEE Transactions on Intelligent Transportation Systems, 25(3), pp. 2733-2747. https://doi.org/10.1109/TITS.2023.3239599