A Comprehensive Model for Evaluating the Performance of the National Research and Technology System Based on Composite Indicators

Document Type : Original Article

Authors

1 Department of Public Administration, Faculty of Economics and Administrative Sciences, University of Sistan and Baluchestan, Zahedan, Iran

2 Department of Public Administration, Faculty of Economics and Administrative Sciences, University of Sistan and Baluchestan, Zahedan, Iran.

3 Department of Higher Education Management, Board of Trustees Center, Ministry of Science, Research and Technology, Tehran, Iran

10.22034/jstp.2026.12344.2061

Abstract

Evaluating the performance of a national research and technology system cannot be limited to measures such as the number of publications, citations, and patents, given the multiplicity of actors, heterogeneity of data, and need for accountability regarding the allocation of public resources. This study aimed to develop and validate a comprehensive, context-specific model for evaluating the performance of Iran’s national research and technology system by integrating foundational capacities, the quality of governance and processes, scientific and technological outputs, and economic, social, and problem-oriented impacts within a coherent framework. The study employed an exploratory sequential mixed-methods design. In the qualitative phase, an initial set of indicators was extracted through a meta-synthesis of 52 academic and policy sources. The indicators were subsequently screened and contextualized through two Delphi rounds involving experts in science, technology, and innovation policy. In the quantitative phase, the relative importance of the dimensions and components was determined using the Best–Worst Method based on the judgments of 14 experts. The findings indicated that the final model comprised four dimensions—enablers, governance and processes, outputs, and impacts—along with 12 components and 36 operational indicators. Governance and processes had the highest weight (0.298), followed by impacts (0.247). Addressing national challenges was also identified as the most important component of the impact dimension. The study’s novelty lies in operationalizing and weighting governance and processes within a comprehensive framework, clarifying their position alongside enablers, outputs, and impacts, and linking research and technology evaluation to priority national challenges and the needs of executive organizations. The proposed model can provide a basis for establishing a national performance-monitoring system, revising the evaluation indicators of executive organizations, strengthening performance-based budgeting, and directing research resources toward addressing national challenges.

Keywords

Main Subjects

References

[1] Audretsch, D. B., & Belitski, M. (2020). The Role of R&D and Knowledge Spillovers in Innovation and Productivity. European Economic Review, 123, 103391. https://doi.org/10.1016/j.euroecorev.2020.103391
[2] Ding, Y., Yin, F., Chin, L., Zhou, K., Taghizadeh-Hesary, F., & Li, Y. (2025). Can Government R&D Expenditure Promote Innovation? New Evidence From 37 OECD Countries. Technological and Economic Development of Economy, 31(2), 572–596. https://doi.org/10.3846/tede.2024.22293
[3] Huang, Y., Li, S., Xiang, X., & Huang, L. (2024). Analyzing the Configuration of the National Innovation System for Innovation Capability: Evidence From Global Innovation Index Reports. Humanities and Social Sciences Communications, 11, 1266. https://doi.org/10.1057/s41599-024-03792-x
[4] Erzurumlu, S. S., Erzurumlu, Y. O., & Yoon, Y. (2022). National Innovation Systems and Dynamic Impact of Institutional Structures on National Innovation Capability: A Configurational Approach With the OKID Method. Technovation, 114, 102552. https://doi.org/10.1016/j.technovation.2022.102552
[5] Coalition for Advancing Research Assessment. (2022). Agreement on Reforming Research Assessment. CoARA. https://doi.org/10.5281/zenodo.13480728
[6] Global Young Academy, InterAcademy Partnership, & International Science Council. (2023). The Future of Research Evaluation: A Synthesis of Current Debates and Developments. Global Young Academy. https://doi.org/10.24948/2023.06
[7] Haddad, C. R., & Bergek, A. (2023). Towards an Integrated Framework for Evaluating Transformative Innovation Policy. Research Policy, 52(2), 104676. https://doi.org/10.1016/j.respol.2022.104676
[8] Larrue, P., Tõnurist, P., & Jonason, D. (2024). Monitoring and Evaluation of Mission-Oriented Innovation Policies: From Theory to Practice (OECD Science, Technology and Industry Working Papers No. 2024/09). OECD Publishing. https://doi.org/10.1787/5e4c3204-en
[9] Garengo, P., & Sardi, A. (2021). Performance Measurement and Management in the Public Sector: State of the Art and Research Opportunities. International Journal of Productivity and Performance Management, 70(7), 1629–1654. https://doi.org/10.1108/IJPPM-03-2020-0102
[10] Lapuente, V., & Van de Walle, S. (2020). The Effects of New Public Management on the Quality of Public Services. Governance, 33(3), 461–475. https://doi.org/10.1111/gove.12502
[11] Wang, H., & Ran, B. (2023). Network Governance and Collaborative Governance: A Thematic Analysis on Their Similarities, Differences, and Entanglements. Public Management Review, 25(6), 1187–1211. https://doi.org/10.1080/14719037.2021.2011389
[12] van den Oord, S., Kenis, P., Raab, J., & Cambré, B. (2023). Modes of Network Governance Revisited: Assessing Their Prevalence, Promises, and Limitations in the Literature. Public Administration Review, 83(6), 1564–1598. https://doi.org/10.1111/puar.13736
[13] Jiménez-Fernández, E., & Ruiz-Martos, M. J. (2020). Review of Some Statistical Methods for Constructing Composite Indicators. Studies of Applied Economics, 38(1). https://doi.org/10.25115/eea.v38i1.3002
[14] Mazziotta, M., & Pareto, A. (2022). Composite Indices Construction: The Performance Interval Approach. Social Indicators Research, 161(2), 541–551. https://doi.org/10.1007/s11205-020-02336-5
[15] Alqararah, K. (2023). Assessing the Robustness of Composite Indicators: The Case of the Global Innovation Index. Journal of Innovation and Entrepreneurship, 12, 61. https://doi.org/10.1186/s13731-023-00332-w
[16] World Intellectual Property Organization. (2024). Global Innovation Index 2024: Unlocking the Promise of Social Entrepreneurship (17th ed.). World Intellectual Property Organization. https://doi.org/10.34667/tind.50062
[17] European Commission, Directorate-General for Research and Innovation. (2025). European Innovation Scoreboard 2025. Publications Office of the European Union. https://doi.org/10.2777/3239776
[18] Ghazinouri, S. S., & Farazkish, M. (2018). National Evaluation Model of Science, Technology and Innovation Based on Efficiency, Effectiveness and Utility Indicators. Strategic Studies of Public Policy, 8(27), 205–229 [In Persian]. https://sspp.iranjournals.ir/article_30761.html
[19] Saadabadi, A. A., Rahimirad, Z., & Fartash, K. (2023). Developing a Comprehensive Performance Evaluation Model for Regional Innovation Systems: A Case Study of the Yazd Province Special Science and Technology Zone. Regional Planning, 13(51), 91–106. https://doi.org/10.30495/jzpm.2021.28081.3899 [In Persian].
[20] Shirzadgan, A., Safdari-Ranjbar, M., & Alizadeh, P. (2024). Investigating Systemic Failures of Iran’s Innovation System: A Structural–Functional Analysis Approach. Management Improvement, 18(4), 112–138. https://doi.org/10.22034/jmi.2025.494876.3162 [In Persian].
[21] Alizadeh, P., & Nasiri, M. (2025). Technology Governance Structure in Iran: Analysis of Institutional Challenges and Policy Requirements for Improving Efficiency Based on the Content Analysis of Legal Documents. Journal of Knowledge Governance, 3(6), 88–112. https://doi.org/10.22034/jokog.2025.509483.1066 [In Persian].
[22] Farazkish, M. (2024). An Analysis of the Success of the Sixth National Development Plan From the Perspective of Policy Evaluation: An Inquiry Into the Field of Research and Technology. Public Policy, 10(4), 10–36 [In Persian]. https://doi.org/10.22059/jppolicy.2024.99824
[23] Shirali, A. (2025). Identification and Evaluation of Science, Technology and Innovation Indicators in the Policy-Making Domain From Experts’ Perspective. Research Journal of Scientometrics, 11(1), 261–280. https://doi.org/10.22070/rsci.2024.17963.1676 [In Persian].
[24] Mohammadkhani Ghiasvand, H., & Mohammadaliha, M. R. (2020). Identifying and Prioritizing Macro Indicators of Innovation Governance. Strategic Management Studies, 11(43), 113–138. https://doi.org/20.1001.1.22286853.1399.11.43.7.0 [In Persian].
[25] Krogh, A. H., & Triantafillou, P. (2024). Developing New Public Governance as a Public Management Reform Model. Public Management Review, 26(10), 3040–3056. https://doi.org/10.1080/14719037.2024.2313539
[26] Supreme Council of Science, Research and Technology. (2017). Executive Bylaw of the National Science, Technology and Innovation Monitoring and Evaluation System (Approved at the 20th session, February 1, 2017). Secretariat of the Supreme Council of Science, Research and Technology [In Persian]. https://www.atf.gov.ir/
[27] Supreme Council of the Cultural Revolution. (2010). Comprehensive Scientific Map of the Country, With Subsequent Amendments and Additions. Supreme Council of the Cultural Revolution [In Persian]. https://qavanin.ir/
[28] Duarte, M. P., & Carvalho, F. M. P. O. (2025). The Measurement of Innovation: A Systematic Review and Bibliometric Analysis of Global Innovation Index Research. Publications, 13(3), 31. https://doi.org/10.3390/publications13030031
[29] Todorov, L., Shopova, M., Panteleeva, I. M., & Todorova, L. (2024). Innovation Metrics: A Critical Review. Economies, 12(12), 327. https://doi.org/10.3390/economies12120327
[30] Fartash, K., Safdari-Ranjbar, M., & Ghorbani, A. (2023). Analysis of the Functions of the Ministry of Science, Research and Technology in the National Innovation System: A Mixed-Methods Study. Strategic Studies of Public Policy, 13(48), 142–170 [In Persian]. https://doi.org/10.22034/sspp.2023.2011078.3486
[31] Alizadeh, P., Safdari-Ranjbar, M., & Mohajeri, A. (2023). Designing a Policy Mix to Strengthen the Role of the Ministry of Science, Research and Technology in the National Innovation System. Management Improvement, 17(2), 88–114. https://doi.org/10.22034/jmi.2023.399995.2970 [In Persian].
[32] Khan-Mohammadi, H., Hajian-Heydari, M., & Sadeghi-Kia, M. (2026). A Systematic Review of Innovation System Functions. Science and Technology Policy Letter, 15(1), 124–140 [In Persian].
[33] Sadeghi-Kia, M. (2021). Mission-Oriented Innovation Policy: Challenges and Opportunities. Science and Technology Policy Letter, 11(1), 113–128 [In Persian].
[34] Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. BMJ, 372, Article n71. https://doi.org/10.1136/bmj.n71
[35] Lawshe, C. H. (1975). A Quantitative Approach to Content Validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
[36] Jabbari, A., Najafpour, Z., Ourang, S., & Cheraghi, M. (2025). Developing and Validating Key Performance Indicators for Breast, Cervical, and Colorectal Cancer Screening Programs: A Literature Review and Delphi Survey. Frontiers in Public Health, 13, 1491226. https://doi.org/10.3389/fpubh.2025.1491226
[37] Liang, F., Brunelli, M., & Rezaei, J. (2020). Consistency Issues in the Best Worst Method: Measurements and Thresholds. Omega, 96, 102175. https://doi.org/10.1016/j.omega.2019.102175
[38] Fatemi, M. (2026). Research and Innovation Policy Design: Policies, Institutions, and Interest Mediation. Journal of Science and Technology Policy, 19(1), 89–108. https://doi.org/10.22034/jstp.2026.12301.2048 [In Persian].
Journal of Science and Technology Policy
Volume 19, Issue 3 - Serial Number 71
September 2026
Pages 21-43

Files

Share

How to cite

Statistics