Comparative Evaluation of Supervised Machine Learning Models for IoT Botnet Detection using Random Forest, XGBoost, and ANN
Main Article Content
Abstract
This research presents a comparative experimental study of machine learning models for botnet attack detection in Internet of Things network using N-BaIoT dataset. The dataset consists of benign traffic and malicious traffic generated by Mirai and BASHLITE botnets family. A total of 115 traffic features are used for classification. The study compares three representative models of learning, namely Random Forest as a classical machine learning classifier, XGBoost as an advanced model of ensemble learning and fully connected Artificial Neural Network as a deep learning classifier. The evaluation of the models was done looking for performance metrics like accuracy, precision, recall, F1-score etc. As per the experiments conducted, the developed system achieved the best which is XGBoost which produced an accuracy rate of 99.12%, a precision of 98.98%, a 99.26% recall, 99.12% F1-score, a 0.88% false alarm rate, and an AUC which is 0.998. The model also achieved a remarkable inference time of 0.018 ms per sample, outperforming Random Forest with 0.021 ms and ANN at 0.035 ms. These results indicate that XGBoost provides the most effective balance between detection accuracy and computational efficiency, making it a suitable candidate for near-real-time IoT botnet detection at gateway or edge-monitoring levels.
Downloads
Article Details
Section
How to Cite
References
Abbasi, M., Farokhnia, A., Bahreinimotlagh, M. and Roozbahani, R., 2021. A hybrid of Random Forest and Deep Auto-Encoder with support vector regression methods for accuracy improvement and uncertainty reduction of long-term streamflow prediction. Journal of Hydrology, 597, P. 125717. https://doi.org/10.1016/j.jhydrol.2020.125717
Alharbi, A., Alosaimi, W., Alyami, H. and Rauf, H.T., 2021. Botnet attack detection using local global best bat algorithm for industrial Internet of Things. Electronics, 10(11), P. 1341. https://doi.org/10.3390/electronics10111341
Alqahtani, A., Alsulami, A.A., Alqahtani, N., Alturki, B. and Alghamdi, B.M., 2024. A comprehensive security framework for asymmetrical IoT network environments to monitor and classify cyberattack via machine learning. Symmetry, 16(9), P. 1121. https://doi.org/10.3390/sym16091121 .
Baker, Q.B. and Samarneh, A., 2024. Feature selection for IoT botnet detection using equilibrium and Battle Royale Optimization. Computers & Security, 147, P. 104060. https://doi.org/10.1016/j.cose.2024.104060 .
Cao, Y., Wang, Z., Ding, H., Zhang, J. and Li, B., 2023. IoT botnet attacks detection and classification based on ensemble learning. In International Symposium on Artificial Intelligence and Robotics, pp. 45-55. Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-99-9109-9_5 .
Catillo, M. and Pecchia, A., 2022. Botnet detection in the Internet of Things through all-in-one deep autoencoding. In: Proceedings of the 17th International Conference on Availability, Reliability and Security (ARES ’22). https://doi.org/10.1145/3538969.3544460 .
Catillo, M., Pecchia, A. and Villano, U., 2023. A deep learning method for lightweight and cross-device IoT botnet detection. Applied Sciences, 13(2), P. 837. https://doi.org/10.3390/app13020837 .
Cunha, A.A., Borges, J.B. and Loureiro, A., 2022. Classification of botnet attacks in IoT using a convolutional neural network. In: MSWiM ’22: Proceedings of the 25th International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems. https://doi.org/10.1145/3551661.3561374 .
Dietterich, T.G., 1998. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), pp. 1895–1923. https://doi.org/10.1162/089976698300017197 .
Fawcett, T., 2006. An introduction to ROC analysis. Pattern Recognition Letters, 27(8), pp. 861–874. https://doi.org/10.1016/j.patrec.2005.10.010 .
Garg, U., Kumar, S. and Mahanti, A., 2024. IMTIBOT: An intelligent mitigation technique for IoT botnets. Future Internet, 16(6), P. 212. https://doi.org/10.3390/fi16060212 .
Gelenbe, E. and Nakip, M., 2022. Traffic based sequential learning during botnet attacks to identify compromised IoT devices. IEEE Access, 10, pp. 126536–126549. https://doi.org/10.1109/ACCESS.2022.3226700 .
Gelgi, M., Guan, Y., Arunachala, S., Samba Siva Rao, M. and Dragoni, N., 2024. Systematic literature review of IoT botnet DDoS attacks and evaluation of detection techniques. Sensors, 24(11), P. 3571. https://doi.org/10.3390/s24113571 .
Hashim, W.A., Ali, S.S.M., Al-Khawaldeh, A. and Al-Shammri, F.K., 2024, December. Botnet Detection Using Hybrid Methods. In 2024 International Jordanian Cybersecurity Conference (IJCC), pp. 21-27. IEEE. https://doi.org/10.1109/IJCC64742.2024.10847282
Hossain, M.A. and Islam, M.S., 2025. A novel feature selection-driven ensemble learning approach for accurate botnet attack detection. Alexandria Engineering Journal, 118, pp. 261–277. https://doi.org/10.1016/j.aej.2025.01.042 .
Kalakoti, R., Bahsi, H. and Nomm, S., 2024. Improving IoT security with explainable AI: Quantitative evaluation of explainability for IoT botnet detection. IEEE Internet of Things Journal, 11(10), pp. 18237–18254. https://doi.org/10.1109/JIOT.2024.3360626 .
Kamal, H. and Mashaly, M., 2025. Robust intrusion detection system using an improved hybrid deep learning model for binary and multi-class classification in IoT networks. Technologies, 13(3), P. 102. https://doi.org/10.3390/technologies13030102 .
Kaufman, S., Rosset, S., Perlich, C. and Stitelman, O., 2012. Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data, 6(4), Article 15. https://doi.org/10.1145/2382577.2382579 .
Khan, M.R.A., Barnawi, A.Y., Munir, A., Alsalman, Z. and Sanunga, D.M.S., 2025. Lightweight quantized XGBoost for botnet detection in resource-constrained IoT networks. IoT, 6(4), P. 70. https://doi.org/10.3390/iot6040070 .
Kim, J., Shim, M., Hong, S., Shin, Y. and Choi, E., 2020. Intelligent detection of IoT botnets using machine learning and deep learning. Applied Sciences, 10(19), P. 7009. https://doi.org/10.3390/app10197009 .
Kumar, A., Shridhar, M., Swaminathan, S. and Lim, T.J., 2022. Machine learning-based early detection of IoT botnets using network-edge traffic. Computers & Security, 117, P. 102693. https://doi.org/10.1016/j.cose.2022.102693 .
Lefoane, M., Ghafir, I., Kabir, S. and Awan, I.-U., 2025. Internet of Things botnets: A survey on artificial intelligence-based detection techniques. Journal of Network and Computer Applications, 237, P. 104110. https://doi.org/10.1016/j.jnca.2025.104110 .
Li, J., Othman, M.S., Chen, H. and Yusuf, L.M., 2024. Optimizing IoT intrusion detection system: Feature selection versus feature extraction in machine learning. Journal of Big Data, 11, P. 36. https://doi.org/10.1186/s40537-024-00892-y .
Liu, X. and Du, Y., 2023. Towards effective feature selection for IoT botnet attack detection using a genetic algorithm. Electronics, 12(5), P. 1260. https://doi.org/10.3390/electronics12051260 .
Malik, K., Rehman, F., Maqsood, T., Mustafa, S., Khalid, O. and Akhunzada, A., 2022. Lightweight Internet of Things botnet detection using one-class classification. Sensors, 22(10), P. 3646. https://doi.org/10.3390/s22103646 .
Meidan, Y., Bohadana, M., Mathov, Y., Mirsky, Y., Breitenbacher, D., Shabtai, A. and Elovici, Y., 2018. N-BaIoT—Network-based detection of IoT botnet attacks using deep autoencoders. IEEE Pervasive Computing, 17(3), pp. 12–22. https://doi.org/10.1109/MPRV.2018.03367731 .
Nazir, A., He, J., Zhu, N., Wajahat, A., Ma, X., Ullah, F., Qureshi, S. and Pathan, M.S., 2023. Advancing IoT security: A systematic review of machine learning approaches for the detection of IoT botnets. Journal of King Saud University - Computer and Information Sciences, 35(9), P. 101820. https://doi.org/10.1016/j.jksuci.2023.101820 .
Powers, D.M.W., 2011. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies, 2(1), pp. 37–63.
Saied, M. and Guirguis, S., 2025. Explainable artificial intelligence for botnet detection in internet of things. Scientific Reports, 15, P. 7632. https://doi.org/10.1038/s41598-025-90420-6 .
Serhane, A., Ibrahimi, K., Hamzaoui, E.-M., Jouhari, M. and others, 2025. IoT intrusion detection using machine learning classifiers and PCA dimensionality reduction for N-BaIoT dataset. In: ICC 2025 - IEEE International Conference on Communications. https://doi.org/10.1109/ICC52391.2025.11161305 .
Sokolova, M. and Lapalme, G., 2009. A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), pp. 427–437. https://doi.org/10.1016/j.ipm.2009.03.002 .
Thein, T.T., Shiraishi, Y. and Morii, M., 2024. Personalized federated learning-based intrusion detection system: Poisoning attack and defense. Future Generation Computer Systems, 153, pp. 182–192. https://doi.org/10.1016/j.future.2023.10.005
Wang, Z., Li, J., Yang, S., Luo, X., Li, D. and Mahmoodi, S., 2024. A lightweight IoT intrusion detection model based on improved BERT-of-Theseus. Expert Systems with Applications, 238, P. 122045. https://doi.org/10.1016/j.eswa.2023.122045 .
Wardana, A.A., Kołaczek, G., Warzyński, A. and Sukarno, P., 2024. Ensemble averaging deep neural network for botnet detection in heterogeneous Internet of Things devices. Scientific Reports, 14, P. 3878. https://doi.org/10.1038/s41598-024-54438-6
Wu, Y., He, X. and Chen, X., 2022. IoT-botnet traffic detection based on deep forest. In: 2022 IEEE 22nd International Conference on Communication Technology (ICCT), pp. 1388–1393. https://doi.org/10.1109/ICCT56141.2022.10072774 .