Review and Analysis of Smart City Evaluation Frameworks

Main Article Content

Enass Mohammed
Meervat R. Altaie

Abstract

Over the past 10 years, there has been a surge in the study and implementation of smart city (SC) technology. A detailed analysis of the literature evaluations that have already been published on SCs reveals a deficiency of studies that categorize the literature into distinct themes and determine the most and least popular topics based on the number of peer-reviewed research papers that fall under each theme. In an attempt to close this gap, a descriptive literature evaluation of peer-reviewed papers on SCs has been carried out for this work. Over the past 10 years, there has been a surge in the study and implementation of smart city (SC) technology. Based on the number of peer-reviewed research articles that fall under each subject, a detailed analysis of the literature evaluations that have already been published on SCs reveals a lack of studies that classify the literature into different themes and identify the most and least popular topics. The investigation shows that although various frameworks provide insightful information, there is sometimes a lack of standardization, which complicates comparisons. Furthermore, the majority of frameworks fall short in addressing issues of social justice and public involvement in the assessment process. The study emphasizes the requirement for a single framework that guarantees inclusivity and integrates a variety of criteria.

Article Details

Section

Articles

How to Cite

“Review and Analysis of Smart City Evaluation Frameworks” (2025) Journal of Engineering, 31(4), pp. 62–81. doi:10.31026/j.eng.2025.04.05.

References

Ajmal, T.S., Arya, S.B., Maurya, P., and Shariff, S.M., 2022. Effect of hydrodynamics and laser surface melting on erosion-corrosion of X70 steel pipe elbow in oilfield slurry. International Journal of Pressure Vessels and Piping, 194, P. 104687. https://doi.org/10.1016/j.ijpvp.2022.104687

Albino, V., Berardi, U., and Dangelico, R.M., 2015. Smart cities: Definitions, dimensions, performance, and initiatives. Journal of Urban Technology, 22(1), pp. 3–21. https://doi.org/10.1080/10630732.2014.942092

Alsamhi, S.H., Ma, O., Ansari, M.S., and Almalki, F.A., 2019. Survey on collaborative smart drones and Internet of Things for improving smartness of smart cities. IEEE Access, 7, pp. 128125–128152. https://doi.org/10.1109/ACCESS.2019.2933494

Altaie, M.R., and Onyelowe, K., 2024. Identifying failure factors in the implementation of residential complex projects in iraq. Journal of Engineering, 30(02), pp. 1-15. https://doi.org/10.31026/j.eng.2024.02.01

Angelakoglou, K., Nikolopoulos, N., Giourka, P., Svensson, I.L., Tsarchopoulos, P., Tryferidis, A., and Tzovaras, D., 2019. A methodological framework for the selection of key performance indicators to assess smart city solutions. Smart Cities, 2, pp. 269–306. https://doi.org/10.3390/smartcities2020018

Antasari, C. and Putri, A.R.F., 2022. Dissemination of public information through online media: Study on the vision and mission of the Mayor of Palu Hadianto Rasyid. Legal, 11(4), pp. 2363–2371. https://doi.org/10.35335/legal.xx.xx

Aragão, F.V., Chiroli, D.M.G., Zola, F.C., Aragão, E.V., Marinho, L.H.N., Correa, A.L.C. and Colmenero, J.C., 2023. Smart cities maturity model—A multicriteria approach. Sustainability, 15(8), P. 6695. https://doi.org/10.3390/su15086695.

Attaran, H., Kheibari, N. and Bahrepour, D., 2022. Toward integrated smart city: A new model for implementation and design challenges. GeoJournal, 87, pp. 511–526. https://doi.org/10.1007/s10708-021-10560-w

Baig, Z.A., Szewczyk, P., Valli, C., Rabadia, P., Hannay, P., Chernyshev, M., Johnstone, M., Kerai, P., Ibrahim, A. and Sansurooah, K., 2017. Future challenges for smart cities: Cyber-security and digital forensics. Digital Investigation, 22, pp. 3–13. https://doi.org/10.1007/s10708-021-10560-w

Bhattacharya, T.R., Bhattacharya, A., Mclellan, B., and Tezuka, T., 2020. Sustainable smart city development framework for developing countries. Urban Research and Practice, 13(2), pp. 180–212. https://doi.org/10.1080/17535069.2019.1641137

Bibri, S.E. and Krogstie, J., 2017. Smart sustainable cities of the future: An extensive interdisciplinary literature review. Sustainable Cities and Society, 31, pp. 183–212. https://doi.org/10.1016/j.scs.2017.01.003

Bluman, G.W. and Cole, J.D., 2012. Similarity methods for differential equations. Springer Science and Business Media, 13. https://doi.org/10.1007/978-1-4612-6394-4

Capdevila, I. and Zarlenga, M.I., 2015. Smart city or smart citizens? The Barcelona case. SSRN Electronic Journal, 8, pp. 266–282. https://doi.org/10.2139/ssrn.2601343

Caragliu, A. and Del Bo, C.F., 2015. Do smart cities invest in smarter policies? Learning from the past, planning for the future. Social Science Computer Review, 34, pp. 657–672. https://doi.org/10.1177/0894439314560506

Cassandras, C.G., 2016. Smart cities as cyber-physical social systems. Engineering, 2, pp. 156–158. https://doi.org/10.1016/j.eng.2016.05.003.

Cavada, M., Hunt, D. and Rogers, C.D.F., 2017. The role of infrastructure in smart cities. In Conference Proceedings, International Symposium for Next Generation Infrastructure. ISNGI London, pp. 72–79.

Cheng, Q., and Li, X., 2020. Construction of smart city evaluation system based on big data. In IOP Conference Series: Materials Science and Engineering (Vol. 740, No. 1, p. 012164). IOP Publishing.‏ https://doi.org/10.1088/1757-899X/740/1/012164.

Borgman, H. P., Heier, H., B2012. Understanding smart cities: An integrative framework. In Proceedings of the 45th Hawaii International Conference on System Sciences (HICSS). Maui, HI, 4–7 January, pp.2289–2297. New York: IEEE. https://doi.org/10.1109/HICSS.2012.465.

Chu, Z., Cheng, M. and Yu, N.N., 2021. A smart city is a less polluted city. Technological Forecasting and Social Change .72, P. 121037. https://doi.org/10.1016/j.techfore.2021.121037

Creutzig, F., Jochem, P., Edelenbosch, O.Y., Mattauch, L., Vuuren, D.P., McCollum, D. and Minx, J., 2015. Transport: A roadblock to climate change mitigation? Science, 350(6263), pp. 911–912. https://doi.org/10.1126/science.aab3294.

Dameri, R.P., 2013. Searching for smart city definition: A comprehensive proposal. International Journal of Computers and Technology, 11(5), pp. 2544–2551. https://doi.org/10.24297/ijct.v11i5.1362

Dashkevych, O. and Portnov, B.A., 2022. Criteria for smart city identification: A systematic literature review. Sustainability, 14(15), p.4448. https://doi.org/10.3390/su14154448.

De Guimarães, J.C.F., Severo, E.A., Felix Júnior, L.A., Batista Da Costa, W.P.L. and Salmoria, F.T., 2020. Governance and quality of life in smart cities: Towards sustainable development goals. Journal of Cleaner Production, 253, P. 119926. https://doi.org/10.1016/j.jclepro.2019.119926.

De Lima, R., Rocha Júnior, V., Dias, F.T., Honorato Filho, M. and Salgueirinho, J.B., Structuring indicators for the development of a smart city plan. European Commission, 2023. Smart Cities.

Fernandez-Anez, V. 2016. Stakeholders approach to smart cities: A survey on smart city definitions. In: Smart Cities: First International Conference, Smart-CT 2016, Málaga, Spain, Proceedings 1 (pp. 157–167). Springer . http://dx.doi.org/10.1007/978-3-319-39595-1_16

Gharaibeh, A., Salahuddin, M.A., Hussini, S.J., Khreishah, A., Khalil, I., Guizani, M. and Al-Fuqaha, A., 2017. Smart cities: A survey on data management, security, and enabling technologies. IEEE Communications Surveys and Tutorials, 19, pp. 2456–2501. https://doi.org/10.1109/COMST.2017.2691146

Govindarajan, R. and Dhanavandan, S., 2020. Usage of library collection, services and assistance among ophthalmologists—An Opinion Survey. International Journal of Knowledge Content Development and Technology, 10(2), pp. 39–49. http://dx.doi.org/10.5865/IJKCT.2020.10.2.039

Gravetter, F.J., and Wallnau, L.B., 2016. Statistics for the behavioral science ,Cengage Learning. http://dx.doi.org/10.2307/1268475

Gupta, P., Chauhan, S. and Jaiswal, M.P., 2019. Classification of smart city research—a descriptive literature review and future research agenda. Information Systems Frontiers, 21(3), pp. 661–685. https://doi.org/10.1007/s10796-018-9844-7

Haque, A.K.M.B., Bhushan, B. and Dhiman, G., 2021. Performance assessment of smart city implementation: A fuzzy-based approach. Sustainable Cities and Society, 74, pp. 103316. https://doi.org/10.1016/j.scs.2021.103316.

He, K. and Wu, J., 2020. Smart city and sustainable urban development: A review of recent research. Sustainability, 12(14), P. 5801. https://doi.org/10.3390/su12145801

Hollands, R.G., 2008. Will the real smart city please stand up? Intelligent, progressive or entrepreneurial? City, 12(3), pp. 303–320. https://doi.org/10.1080/13604810802479126.

Huang, C.D., Wu, D.D., Liu, H.K., Zhang, L. and Chen, Q., 2017. Enhancing smart city ORCIDO with the Internet of Things. Sustainability, 9(11), P. 2071. https://doi.org/10.3390/su9112071.

Huang, Y., Peng, H., Zhong, A., Zhou, Z., and Sofi, M., 2020. The application of the latest technologies for smart city construction in China. Preprints. pp. 1–18. https://doi.org/10.20944/preprints202007.0116.v1

International Organization for Standardization. (ISO 37120 ), 2014. Sustainable Development of Communities: Indicators for City Services and Quality of Life. ISO.‏

Lytras, M.D., Visvizi, A., Torres-Ruiz, M., Damiani, E., Jin, P., 2020. IEEE access special section editorial: Urban computing and well-being in smart cities: Services, applications, policymaking considerations. IEEE Access, 8, pp. 72340-72346.. https://doi.org/10.1109/ACCESS.2020.2988125

Jin, L., Xiang, Y., Zhou, Y. and Liu, J., 2019. A review of the smart city concept and its applications in China. Sustainable Cities and Society, 45, pp. 547–557. https://doi.org/10.1016/j.scs.2018.12.037.

Jin, X., Zhang, L., Li, X., Li, Y., Xu, W. and Zhang, J., 2022. A hybrid model for smart city risk assessment based on data mining and machine learning. Systems, 10(7), P. 220. https://doi.org/10.3390/systems10070220.

Kaur, H. and Garg, P., 2019. Urban sustainability assessment tools: A review. Journal of Cleaner Production, 210, pp. 146–158. https://doi.org/10.1016/j.jclepro.2018.10.141.

Kitchin, R., 2014. Big data, new epistemologies and paradigm shifts. Big Data and Society, 1(1), pp. 1–12. https://doi.org/10.1177/2053951714528481.

Li, X., Li, X., Xu, W., Zhang, L. and Zhang, J., 2021. An integrated model for evaluating smart city development performance: Evidence from China. Sustainable Cities and Society, 73, P. 103133. https://doi.org/10.1016/j.scs.2021.103133.

Liu, Z., Yu, H. and Xu, L., 2022. The role of smart cities in sustainable development: A systematic literature review. Journal of Cleaner Production, 359, P. 132003. https://doi.org/10.1016/j.jclepro.2022.132003.

Mills, D., Pudney, S., Pevcin, P. and Dvorak, J., 2021. Evidence-based public policy decision-making in smart cities: Does extant theory support achievement of city sustainability objectives? Sustainability, 14(1), P. 3. https://doi.org/10.3390/su14010003

Mohanty, S.P., Choppali, U. and Kougianos, E., 2016. Everything you wanted to know about smart cities: The Internet of things is the backbone. IEEE Consumer Electronics Magazine, 5(3), pp. 60–70. https://ieeexplore.ieee.org/document/7469382

Mourad, K., 2021. Challenges and opportunities for implementing smart cities. Journal of Urban Management, 10(2), pp. 14–24. https://doi.org/10.1016/j.jum.2021.01.004.

Murphy, M.E., Perera, S., and Heaney, G., 2015. Innovation management model: A tool for sustained implementation of product innovation into construction projects. Construction Management and Economics, 33(3), pp. 209–232 https://doi.org/10.1080/01446193.2015.1031684.

Myeong, S., Jung, Y., and Lee, E., 2018. A study on determinant factors in smart city development: An analytic hierarchy process analysis. Sustainability, 10(8), P. 2606. https://doi.org/10.3390/su10082606.

Nasution, M.I., Sitorus, R.M., Silalahi, D., Siregar, J., Sirait, M.E. and Siregar, J., 2021. Smart city evaluation framework: A case study of smart city implementation in Indonesia. IEEE Access, 9, pp. 42537–42549. https://doi.org/10.1109/ACCESS.2021.3061270.

Onoda, H., 2020. Smart approaches to waste management for post‐COVID‐19 smart cities in Japan. IET Smart Cities, 2(2), pp. 89–94.https://doi.org/10.1049/iet-smc.2020.0063.

Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Moher, D., 2021. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. bmj, 372. https://doi.org/10.1016/j.ijsu.2021.105906.

Pappalardo, L., Greco, S. and Mousseau, V., 2016. A comprehensive review of the smart city concept: Definitions, dimensions, metrics, and practices. Journal of Urban Technology, 23(3), pp. 1–22. https://doi.org/10.1080/10630732.2016.1198731.

Parra, C., Yigitcanlar, T., Sipe, N., Montavon, M. and Kellett, J., 2016. Understanding smart cities: A systematic literature review. Infrastructures, 1(2), pp. 89–104. https://doi.org/10.3390/infrastructures1020089.

Patel, M., Rajgor, M.B. and Jayeshkumar, R.P., 2016. Factors affecting on contractor’s bidding strategy in large scale residential construction. Development, 3(5), pp. 393–399. http://dx.doi.org/10.21090/IJAERD.030561

Patrão, C., Moura, P., and Almeida, A.T., 2020. Review of smart city assessment tools. Smart Cities, 3(4), pp. 1117–1132. https://doi.org/10.3390/smartcities3040055

Pereira, J.P., Gouveia, J.M., Almeida, M.P. and Silva, P.F., 2022. Smart cities: Theory, practice and future. Smart Cities, 5(1), pp. 16–32. https://doi.org/10.3390/smartcities5010002.

Puron-Cid, G., Luna, D.E., Picazo-Vela, S., Gil-Garcia, J.R., Sandoval-Almazan, R. and Luna-Reyes, L.F., 2022. Improving the assessment of digital services in government websites: Evidence from the Mexican State government portals ranking. Government Information Quarterly, 39(1), 101589. https://doi.org/10.1016/j.giq.2021.101589

Sharifi, A., 2019. A critical review of smart city definitions. Cities, 87, pp.103–115. https://doi.org/10.1016/j.cities.2019.01.003.

Sharonova, S. and Avdeeva, E., 2024. Smart education: Social risks and challenges. In IoT, AI, and ICT for educational applications: Technologies to enable education for all. Cham: Springer Nature Switzerland, pp. 99–118. http://dx.doi.org/10.1007/978-3-031-50139-5_5

Shelton, T., Zook, M. and Wiig, A., 2015. The actually existing smart city. Cambridge Journal of Regions, Economy and Society, 8(1), pp. 13–25. http://dx.doi.org/10.1093/cjres/rsu026

Shukla, S., Salve, N., and Kalangade, J. 2023. 8 Smart cities in Asia. Smart Cities in Europe and Asia: Urban Planning and Management for a Sustainable Future, 93.‏ https://doi.org/10.4324/9781003365174-8 .

Syalianda, S.I., and Kusumastuti, R.D., 2021. Implementation of smart city concept: A case of Jakarta smart city, Indonesia. In IOP Conference Series: Earth and Environmental Science (P. 12128). IOP Publishing. https://doi.org/10.1088/1755-1315/12128.

Taddeo, M., 2020. How to build a smart city: From planning to implementation. Technological Forecasting and Social Change, 155, P. 119981. https://doi.org/10.1016/j.techfore.2020.119981.

Torres, J.M., Ferreira, J.C., Casimiro, S., Ferreira, A., Lima, C. and Santos, A., 2022. Smart city development: The state of the art and future directions. Technology in Society, 68, P. 101832. https://doi.org/10.1016/j.techsoc.2022.101832.

Van den Bergh, J., and Viaene, S. 2015 . Key challenges for the smart city: Turning ambition into reality. In 2015 48th Hawaii International Conference on System Sciences (pp. 2385-2394). IEEE.‏ https://doi.org/10.1109/HICSS.2015.642

Wan, S., Lu, J., Fan, P. and Letaief, K.B., 2017. To smart city: Public safety network design for emergency. IEEE Access, 6, pp. 1451–1460. http://dx.doi.org/10.1109/ACCESS.2017.2779137

Wang, J., Wu, Y., Wang, L. and Li, M., 2021. Measuring the performance of smart cities: An empirical study of 35 Chinese cities. Sustainable Cities and Society, 68, P. 102671. https://doi.org/10.1016/j.scs.2021.102671.

Wang, Y., Hu, Z., Chen, Z., Li, Y. and Li, L., 2020. Analyzing the smart city research trend based on bibliometric analysis. Sustainability, 12(9), P. 3771. https://doi.org/10.3390/su12093771.

Wu, H. and Li, W., 2019. A systematic review of smart city definitions, indicators, and applications. Urban Science, 3(2), P. 37. https://doi.org/10.3390/urbansci3020037.

Xie, X., Liu, X., Zheng, Y. and Zhao, Y., 2022. Exploring the development of smart cities: A bibliometric and content analysis. Sustainable Cities and Society, 79, P. 103756. https://doi.org/10.1016/j.scs.2022.103756.

Yang, H., Zhang, Y., Liu, Z., Liu, J. and Yang, X., 2021. Smart city research: A systematic review and future research agenda. Journal of Urban Technology, 28(2), pp. 3–27. https://doi.org/10.1080/10630732.2020.1819835.

Zeng, M., Li, S., Zhang, X. and Zheng, Y., 2022. A novel framework for smart city development based on a smart city maturity model. Sustainable Cities and Society, 78, P. 103587. https://doi.org/10.1016/j.scs.2021.103587.

Zhou, X., Yang, Z., Xu, L., Liu, X., Zhang, Y. and Wang, Y., 2022. Evaluating the performance of smart cities: A novel framework based on balanced scorecard and data envelopment analysis. Journal of Cleaner Production, 334, P. 130211. https://doi.org/10.1016/j.jclepro.2021.130211.

Zou, D., Jiang, X., Yang, Z., Zhang, C., Zhao, X., Li, J. and Sun, L., 2022. Integrating smart city frameworks and indicators: A systematic review and case study. Sustainability, 14(14), P. 8772. https://doi.org/10.3390/su14148772

Similar Articles

You may also start an advanced similarity search for this article.