|Year : 2020 | Volume
| Issue : 1 | Page : 165
Frameworks of performance measurement in public health and primary care system: A scoping review and meta-synthesis
Somayeh Noorihekmat1, Hamed Rahimi2, Mohammad H Mehrolhassani1, Mohamadreza Chashmyazdan3, Ali Akbar Haghdoost4, Seyed Vahid Ahmadi Tabatabaei5, Reza Dehnavieh1
1 Health Services Management Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
2 Students Research Committee, Kerman University of Medical Sciences, Kerman, Iran
3 Ph.D. Student in Medical Library and Information Science, Kerman University of Medical Sciences, Kerman, Iran
4 Modeling in Health Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
5 Social Determinants of Health Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
|Date of Submission||27-Jan-2019|
|Date of Acceptance||29-Aug-2019|
|Date of Web Publication||05-Oct-2020|
Kerman University of Medical Sciences, Haft Baghe Alavi Blvd., Kerman
Source of Support: None, Conflict of Interest: None
Background: This study was conducted to synthesize the evidence on the dimensions of performance appraisal of the public health and primary care system through a scoping review and meta-synthesis. Methods: The review conducted systematically in 2018 with a scoping review approach. To identify pertinent studies, the following electronic databases were systematically searched until December 20, 2017: Cochrane, ISI Web of Science, PubMed, Scopus, Science Direct, and Embase. Reviewing the studies found on the search bases was carried out in three stages by two persons individually. According to refined studies, the data were extracted to meet the objectives and respond to the research questions. The thematic analysis was used to identify and categorize the dimensions of performance measurement. Results: Using this process, 20 studies were eligible for our research. The critical points in measuring the performance of the public health field were classified into eight main domains including leadership and stewardship, funding, resource generation, service delivery, quality, accessibility, efficiency/productivity, and community health status. The differences in measurement frameworks are inevitable. One reason for the differences in the health system performance measurement framework is the differences in the data or data collection, analysis, and reporting. Performance measurement in the field of health, especially primary care, was a multidimensional issue. Conclusions: Each of the main dimensions had several sub-criteria, indicating the broadness and complexity of the performance of first-level care providers. Single-dimensional performance measurement could underpin incorrect policies and decisions.
Keywords: Efficiency, performance, primary health care, public health, quality of care, review
|How to cite this article:|
Noorihekmat S, Rahimi H, Mehrolhassani MH, Chashmyazdan M, Haghdoost AA, Ahmadi Tabatabaei SV, Dehnavieh R. Frameworks of performance measurement in public health and primary care system: A scoping review and meta-synthesis. Int J Prev Med 2020;11:165
|How to cite this URL:|
Noorihekmat S, Rahimi H, Mehrolhassani MH, Chashmyazdan M, Haghdoost AA, Ahmadi Tabatabaei SV, Dehnavieh R. Frameworks of performance measurement in public health and primary care system: A scoping review and meta-synthesis. Int J Prev Med [serial online] 2020 [cited 2020 Oct 19];11:165. Available from: https://www.ijpvmjournal.net/text.asp?2020/11/1/165/297327
| Introduction|| |
Rapid changes in today's world have affected organizations' activities so that the lack of awareness of these changes will lead to exposure to some complicated crises., These rapid and widespread changes have also affected health systems, which are considered as one of the important factors of development and social welfare.,, One of the main consequences of these changes is their impact on people's health. Therefore, it is essential for health service providers to enhance their performance to respond to these changes. For this reason, improving the performance of health systems worldwide is considered by governments, investors, and providers. The key to improve performance is to measure performance. Performance measurement is itself some part of the strategic process of performance management.
Focusing on achieving good and sustainable outcomes, the process of performance management considers internal and external factors and achieves certain strategic and measurable goals. It also empowers organization leaders to do monitoring and find out how to achieve their goals. Performance measurement is a great movement around the world that provides some feedback for learning and can help educate managers about the behavior and function of employees. Meanwhile, regarding international health goals, there is interest in increasing the performance measurement of health systems in different countries (especially in developing countries).
Some of the reasons for the increased interest in performance measurement in health systems are the increase in costs, technological advancements, the growth of elderly populations, health market failures, poor quality and deviations of execution, medical errors and injuries, lack of accountability and responsibility, unfairness, and uncertainty. Therefore, governments should use tools for monitoring and evaluating the systems' performance and making more informed decisions about financial resources, organizations, and health systems policies, as international organizations such as the WHO and the OECD have played an international role in encouraging the measurement of health systems' performance.
Although the issue of health systems evaluation has a long history, thefirst attempts to regularly measure the performance of healthcare systems were started by the WHO 18 years ago. Since then, increased attention has been paid to performance measurement in the organizations that provide health services. So far, many efforts have been made over the past years to create a performance measurement framework with regard to the characteristics of health systems.
Various performance assessment frameworks are designed and used for health systems and their subsets by international organizations and researchers. Some of these models include balanced scorecard, data envelopment Analysis, Donabedian's framework, Starfield's model, health system performance assessment (HSPA) framework, WHO building blocks framework, WHO health systems functions, control knobs framework, systems thinking framework, and health care quality indicators (HCQI) framework of organization for economic co-operation and development (OECD).,,,, The existence of an appropriate and comprehensive evaluation framework for health system performance can be useful as a decision-making tool for policymakers in the health sector.
The lack of an integrated and universally-accepted framework for measuring the performance of health services has led to different studies on the performance of health organizations from different dimensions., Therefore, given the fact that there is no comprehensive agreement on the performance measurement of organizations providing health services, especially primary care, the development and use of a combination of methods, frameworks, and indicators to measure the performance of these organizations can provide a comprehensive and complete perspective of their capabilities., In this regard, the synthesis of the existing knowledge can provide comprehensive evidence to help managers with planning and decision-making. Doing a comprehensive review and summarizing all pertinent studies on a research question, knowledge synthesis interprets the results of those studies in a general framework of evidence. Given that summarizing and issuing the results of research is one of the main objectives of field reviews, this study was conducted to synthesize the evidence on the dimensions of performance appraisal of the public health and primary care system through a scoping review and meta-synthesis.
| Methods|| |
This is a review conducted systematically in 2018 with a scoping review approach. In this study, Arksey and O'Malley's approach (2005), as well as the complementary recommendations of Levac (2010),, were used to conduct a scoping review and identify the dimensions of the performance measurement of public health and primary care.
Data sources and search strategy
Too identify pertinent studies, the following electronic databases were systematically searched until December 20, 2017: Cochrane, ISI Web of Science, PubMed, Scopus, Science Direct and Embase. To ensure that the study was not repeated, the Cochrane database was searched initially. The literature search was carried out by reviewing the references of the selected studies. The researcher also signed up on the above-mentioned sites and enabled the alert option to get informed of the new studies related to the topic. The keywords were selected after an initial search and consultation with the librarian. The search strategies are presented in [Appendix 1]. The search for literature was done by a library specialist. The search results for review and screening were entered into the EndNote v.X7 software by the research team.
Study selection: (Inclusion criteria, screening)
Reviewing the studies found on the search bases was carried out in three stages (title, abstract, and full text) by two persons individually. In the event of disagreement, the final decision was reached through an agreement, but if no agreement was reached, a third-party's comments were used. The inclusion criteria including English language, after 2000, public health and primary health care studies, performance assessment studies, studies to evaluate, the performance of systems or organizations, and use of assessment models in the study. The reason for choosing 2000 as the base year was thefirst efforts for the performance evaluation of health system started at that year.
In addition, exclusion criteria were studies before 2000, studies of other languages, studies on the field of treatment (hospitals, etc.), staff performance assessment studies, studies on patient evaluation and disease and commentary studies, Letter to the editor, Book, and Report.
The studies were screened using the EndNote v.X7 software. Because it was not conventional to do the quality assessment of the studies in scoping reviews, the quality of the selected articles was not assessed in this study.
According to refined studies, the data were extracted to meet the objectives and respond to the research questions. To do so, a preliminary form wasfirst designed and the data from 2 studies were extracted. The form was then reviewed and used for other articles. According to this form, the authors of the article, years, countries, type of study, research objectives, setting, and indices used were extracted. At this point, one of the authors extracted the required data from the selected studies and the second author examined the data. The form was completed in the Microsoft word software for each of the studies.
Synthesis of data
The thematic analysis was used to identify and categorize the dimensions of performance measurement. Thematic analysis is an inductive method for analyzing qualitative data, which analyzes and understands a particular phenomenon by using open coding to create themes and sub-themes. Extracted data were analyzed using MAXQDA-11. In this study process of thematic analysis included 6 steps: (1) Familiarizing with data: transcribing data, reading and rereading the data, and writing down initial ideas; (2) Producing primary codes: coding extracted data among whole data set and collating related data to every code; (3) Searching for themes: collating codes into possible themes and gathering all related data to every possible theme; (4) Reviewing themes: checking if the themes work in relation to the coded extracted and the whole data set, producing a thematic map; (5) Defining and naming themes: ongoing analysis for refining the specifics of every theme and the whole story that the analysis tells, producing clear definitions and names for every theme; and (6) Generating the report: the final analysis of selected extracts and generating a report of the analysis.
In this study, two of the researchers independently categorized them from conceptual similarities. Focusing on resolving the disputes, they compared their categorizations next. The disputed and uncategorized cases were provided to a third person who was asked to deductively group the uncategorized findings using the defined categories and themes. Then, the third reviewer's comments were compared with the previous categorizations. The disagreements were discussed and the results were ultimately grouped. In the end, the results of the analysis were reviewed and confirmed by two other researchers to assess the understandability and co-ordination of the themes and sub-themes.
| Results|| |
In the initial search, 917 articles were found in scientific databases through manual search, and 456 ones were reviewed after excluding repetitive and unrelated articles. In the second phase, 297 studies were reviewed according to their abstracts. Thereby, 198 articles were excluded owing to the lack of the inclusion criteria (99 studies were selected). Finally, reviewing the full texts of the remaining studies indicated that 19 studies were eligible for our research [Figure 1].
Of the 19 studies selected, 5 were multi-country studies. The study population of this studies included OECD countries, low and middle income ountries (LMICs), European Union (EU) countries, 173 countries, and Netherlands and Ontario, Canada. The scope of most of the included studies was the national health system (63%). Scopes of other studies included primary care at national level (2 studies), service provider organization level (2 studies), and primary care in rural areas (2 studies). One study also examined national and international levels simultaneously. A summary of the features of those studies is presented in [Appendix 2].
On the basis of the analysis carried out, the critical points in measuring the performance of the public health field were classified into 8 main domains including leadership and stewardship, funding, resource generation, service delivery, quality, accessibility, efficiency/utilization, and community health status [Table 1].
|Table 1: Taxonomy of critical points to performance measurement in public health and primary care system|
Click here to view
Leadership and stewardship
A total of 7 studies focused on the issue of leadership and stewardship and its subsets. The results showed that in the leadership and stewardship domain, the studies had examined strategic planning (vision, mission, policies, and strategy), control and adjustment approaches, accountability, inter-sectoral coordination, and organizing of health services.
A total of 10 studies focused on the issue of funding and its subsets. The results suggested that fair financing, costs and expenses, revenue collection, accumulation and allocation of financial resources, and protecting financial risks were the components of this dimension.
A total of 7 studies focused on the issue of resource generation and its subsets. The results of this study also indicated that human resources, information, medicine and medical equipment, and growth and learning of the employees had been evaluated as health system resources in previous studies.
A total of 9 studies focused on the issue of service delivery and its subsets. This theme contains adult health services, child health services, antenatal cervices, continuity, coordination, comprehensiveness, and so on.
A total of 14 studies focused on the issue of quality and its subsets. It was found out that several factors had to be considered for quality assessment. In this regard, the researchers addressed a variety of topics for quality assessment, including the quality of care, management quality, errors and losses, safety, accountability, suitability, and service effectiveness.
A total of 11 studies focused on the issue of accessibility and its subsets. The results also showed that accessibility was a multidimensional issue, and the studies had examined it from different perspectives: financial, information, linguistic and physical (Structural) access, service availability/allocation, rural public health system coverage, equality of rural public health services, cultural competence, and acceptability.
A total of 8 studies focused on the issue of efficiency/productivity and its subsets. It was indicated that efficiency/productivity was one of the aspects of health systems performance measurement.
Community health status
A total of 6 studies focused on the issue of community health status and its subsets. The results of the study showed that the researchers examined community health improvement, healthy living, demographic and socioeconomic factors, health determinants, patient and family satisfaction, and family-centered and community-based services for assessing health status.
| Discussion|| |
It was indicated in this study that there were different structures and approaches to assess the performance of primary care. Some of these differences could be owing to the national policies and programs or owing to the priorities and technical differences in the health system of the countries. However, various frameworks experienced by different countries could be valuable to policymakers, health managers, and researchers from other countries. This review attempted to present the dimensions used to evaluate the performance of primary care as a comprehensive set. The results showed that performance measurement in the field of public health, especially primary care, was a multidimensional issue and all its dimensions needed to be considered. Thus, the topics used were classified into 8 main themes, the importance, role, and position of which are discussed in the following.
Leadership and stewardship
The results of this study showed that leadership and stewardship had been introduced into a few performance evaluation frameworks. According to the results, only seven studies had focused on the field of leadership and stewardship in primary care performance measurement., 14, ,,,, However, the WHO has introduced stewardship as one of the key functions of health systems. Health stewardship that had a close relationship with the concepts of leadership and governance, generally refers to the laws and institutions that shape policies, programs, and activities related to health goals. Makuta and O'Hare indicated a positive relationship between the indicators of stewardship and leadership and the health outcomes and performance rate. However, Lagomarsino et al. believed that the existence of effective leadership in the health system was essential for achieving broad health goals. Besides, Pallas et al. argued that management and leadership strategies for improving organizational performance were more important than the strategies such as creating new skills and knowledge or making a new box in the organizational chart because a new paradigm was needed to determine how the groups within an organization should interact with each other to achieve the organizational goals. One of the interventions of this strategy was said to be the creation of monitoring systems to manage the performance of the organization's executives in response to the organizational goals. Hence, to have a comprehensive framework for assessing the performance of health care providers, it is recommended to take leadership and stewardship into consideration.
The results suggested that fair financing, costs and expenses, revenue collection, accumulation and allocation of financial resources, and protecting financial risks were the components of this dimension. Our study also showed that 50% of the studies had focused on the financial dimension of primary care when measuring primary care performance. In their study, Shahri et al. pointed out that financial resource management played an essential role in measuring the performance of health systems. In their opinion, the three functions of financial management were resource collection, resource accumulation, and purchase of services. The WHO also considered these three functions essential to ensure people's access to health services. In a study by Dos Santos et al., the financial dimension was considered as the most influential dimension of public health systems' performance. Because of the financial system and the costs of providing services build the basis for the efficiency of health systems, the financial system must be regularly and effectively evaluated using validated indicators.
For better performance and continuity of health care provision, the primary care system requires a variety of resources, including human, physical (equipment and facilities), and informational resources. Edward et al., who used the balanced scorecard framework for their assessment of health systems and considered the resources such as medications, functional equipment, and training as the components of capacity dimension for providing services. They also evaluated staff as a separate dimension. Some studies, such as the one by Mutale et al. in which the WHO framework was used, evaluated three components of resource production (human resources, health information, and medical equipment) separately.
Human resources can be considered as one of the most important ones; because by utilizing health technologies, it plays a key role in managing and providing services and promoting community health., However, 60% to 80% of the total health care costs are usually manpower costs. Most importantly, quality, efficiency, effectiveness, availability, and acceptability of health services depend on the performance of those who provide these services. Therefore, given the fact that achieving the goals of health systems, especially primary care, depends on the efficient and capable manpower; assessment and improvement of their performance can lead to improved service quality and increased satisfaction of the clients. Mutale et al. pointed out that interventions in the field of human resources would have an incremental effect on leadership and stewardship, provision of services, financial systems, and health information.
Information resources are another important sources in the health sector, so that having a comprehensive health information system can have many potential advantages in terms of financial benefits and improvement of healthcare quality. Furthermore, the existence of a health information management system helps to determine the appropriate indicators to make decisions and improve the performance of the organization. It should also be noted that having a proper information system in the organization will be useful in the process of evaluating and providing the necessary data to measure the performance of the organization.
Assessment of health services delivery was a key dimension in the performance measurement of health organizations. For instance, in the study by Shahri et al., health service delivery with an impact factor of 0.79 was introduced as the most effective determinants of health system performance assessment. According to the conducted analysis, attention was paid to fairness, continuity, integrity, co-ordination, and services for special groups (Like children, mothers, and adults) when assessing the provision of primary care.
Providing health services to the community is the main process in any health care system that is needed to achieve the main goal of the system. However, providing services represents an immediate (primary) output from the health system's inputs, and a fundamental input to the health status of the community. For this reason, providing health services is the backbone of the health system. Therefore, it is also considered as one of the key pillars of performance evaluation frameworks. For example, one of the 6 main dimensions of the performance system designed to manage the provision of primary care services in Afghanistan was the provision of services. Hence, considering the importance of strengthening the delivery of health services to achieve the millennium development goals (MDGs), monitoring, and evaluating it is essential and recommended.
Quality is a very important issue in health systems because poor service quality leads to diseases, disabilities, higher costs, and lower confidence in the health system. This is why the quality of health services is one of the main dimensions of health system performance. The results showed that 60% of the selected studies focused on quality and its subsets. In their study, Pallas et al. reviewed 181 experimental studies of health service providers in countries with middle-toward-down income and suggested that quality had the highest frequency (83%) among performance dimensions. Studies have shown that researchers' viewpoints toward quality are different. For example, in their study, Tawfik Shukor et al. examined the framework of performance measurement system in the Netherlands and Ontario, and called the three dimensions of safety, effectiveness, and accountability as “quality diamonds”. However, Kruk et al. considered quality of care as one of the subsets of effectiveness and classified its indicators into three categories of impact, safety, and continuity.,
The results of the present study showed that the previous studies had addressed effectiveness, safety, suitability, accountability, errors, clinical quality, and management quality for quality assessment. Accountability, which is one of the primary goals of any health system, has some dimensions that are important indicators of health system performance. In their study, Jacob et al. found a correlation between quality and accountability. There are different perspectives on the role of effectiveness in evaluation frameworks, so that in some studies, it was considered as one of the dimensions of service quality.,, However, in the study by Jahanmehr et al., it was evaluated as an individual dimension. Effectiveness has also been used in the frameworks developed by the WHO and the OECD.,
Quality indicators werefirst developed to improve patients' outcomes, but now their information is used for monitoring, management, and policymaking. However, Van Den Berg et al. in the Netherlands referred in their study to the rarity of good information on quality according to which patients and insurers could make their choices. Therefore, quality assessment and monitoring, which can help managers, insurers, and patients to make decisions and improve health systems should be properly considered.
Shahri et al. addressed fair access to health services and included financial, geographical, individual, and structural factors. To examine access to health services, Pallas et al. addressed financial, informational, linguistic (cultural), and physical (structural) access as well as allocation/availability of services. Tian et al. evaluated rural public
health services and considered two issues of coverage and equality. Pasarin et al. and Gauld et al. separately evaluated accessibility in terms of cultural competence and acceptability of services.,
From patients' perspective, timely access to a complete set of required services is one of the characteristics of an effective health system. In this regard, primary care which is the base of any health systems should be available to all people in the community. However, in developing countries, access to basic services is still one of the major barriers to improve people's well-being. Hence, this issue needs to be considered by policymakers and analysts. One way to measure people's access to health services is to incorporate it into the frameworks of health system performance measurement. Accordingly, the results of the present study showed that this issue has been well considered in the performance measurement frameworks, so that 11 studies (55%) reviewed had evaluated accessibility. In general, accessibility includes three components: availability, utility, and timeliness. These studies have addressed various aspects of access including financial, physical, cultural, geographical, structural, and informational access.,,
Timely access to health services is essential for maintaining lives and minimizing suffering and disabilities. Terner et al. also considered the waiting time for receiving immediate care as one of the indicators of access. Thus, it should be evaluated and measured using valid and accurate indicators, and its barriers need to be eliminated according to the measurement results. It should be noted, however, that access measurement requires attention to manpower and service capacity and whether the services are available.
To evaluate efficiency/utilization, Pallas et al. used the ratios of costs to services, staff to services, use of services to income groups, volume of patients or procedures to population health characteristics, and so on. Sun et al. categorized performance indicators in three groups: direct inputs, direct outputs, and underlying factors. Kruk and Freedman addressed efficiency in relation to management, costs and productivity, and financing adequacy.
In health systems, efficiency measurement is usually thefirst step in evaluating the function of the units such as health centers, hospitals, etc. The reason why efficiency measurement in health systems is important is that patient health outcomes can be measured. However, the inefficient use of resources in primary care is related to the number of avoidable hospitalizations. Nevertheless, the results of the study showed that only 40% of the studies reviewed had addressed efficiency in measuring the performance of public
health providers. However, efficiency is one of the major criteria in resource allocation processes. Therefore, monitoring and evaluating the efficiency of public
health systems at different levels can help to better allocate the resources.
Marschall and Flessa stated that the achievement of low-income countries in the health-related MDGs depended on focusing on efficiency. Efficiency analysis plays an important role in empowering policymakers, managers, and health providers to improve the structures, processes, and outcomes of primary care. However, owing to the unclear boundaries of health service provision, it is a challenging task as well. Efficiency is not only one of the dimensions of the performance measurement framework but also one of the main goals of the implementation of the monitoring and evaluation system. Therefore, although performance measurement in public health systems is complex and challenging, its multiple benefits and implications for health systems have made it necessary and inevitable.
What was said above indicates the importance of efficiency in public health systems. Therefore, special attention should be paid to it by health managers and policymakers. However, some studies might have specifically addressed efficiency, but incorporating it into performance measurement frameworks may lead to better policymaking and decision-making. Hence, to have a comprehensive and effective measurement system, the issue of efficiency should be considered and its indicators need to be monitored.
Community health status
The results of the study showed that the researchers examined community health improvement, healthy living, demographic and socioeconomic factors, health determinants, patient and family satisfaction, and family-centered and community-based services for assessing health status. Because supplying, maintaining, and promoting community health are the main goals of health systems, the extent to which these goals are achieved should be evaluated., From the perspective of society, the essential duty of an effective health system is to improve the health status in the country. Besides, the extent to which the services are based on the community or families is considered for the purpose of evaluating community health. Access to community-based primary care is a priority in many countries. Population-based primary care is intended to providefirst-contact healthcare services to ensure the continuity of care, ease of movement across the system, and improved system integration. However, measuring the consequences is difficult, and it is not easy to distinguish the degree of health system participation from other factors. However, as improving the health status is considered a measure for health system effectiveness, it is an important part of performance measurement and should be considered.
Limitations of study
This meta-synthesis by search criteria has been deliberately limited to measuring the performance of public
health and primary care. Minor measurement criteria such as quality, efficiency, etc., were not included in the search strategy of the present study. Moreover, the findings of this meta-synthesis are limited by searched databases, time, and language of publication. There may be some studies in old literature or other languages that were not entered into this analysis.
| Conclusions|| |
According to the results of this study, performance measurement in the field of health, especially primary care, was a multidimensional issue, indicating the widespread and complexity of the performance offirst-level care providers. Single-dimensional performance measurement could underpin incorrect policies and decisions. If the dimensions and indicators of measurement are well-defined and tied to management and policy processes, it can help improve performance. However, the differences in measurement frameworks are inevitable. One reason for the differences in the health system performance measurement framework is the differences in the data or data collection, analysis, and reporting. According to the evidence, the dimensions used depended on the measurement model used, the purpose of measurement, the view of the executives, and the participants in the studies. Finally, the results of this study can provide a comprehensive viewpoint about performance evaluation to managers and help them to design a performance assessment framework.
This research, extracted from the dissertation, was conducted by Mr. Hamed Rahimi as part of the activities required for a Ph.D. degree in health services management at the Kerman University of Medical Sciences. The authors would like to express their sincere gratitude to the Student Research Committee for its financial support, and to the Health Deputy of Kerman University of Medical Sciences for its administrative support.
Financial support and sponsorship
This study was funded by the Students Research Committee of Kerman University of Medical Sciences (Grants No. 97000405).
Conflicts of interest
There are no conflicts of interest.
| References|| |
Team FME. PESTLE analysis. Strategy Skills. free-management-ebooks. 2013. Available from: https://docplayer.net/9768722-Pestle-analysis-strategy-skills-team-fme-www-free-management-ebooks-com-isbn-978-1-62620-998-5.html. [Last accessed 2019 Oct 23].
Amuna YMA, Al Shobaki MJ, Naser SSA. Strategic environmental scanning: An approach for crises management. Int J Inf Technol Electr Eng 2017;6:28-34.
Rajabi F, Esmailzadeh H, Rostamigooran N, Majdzadeh R, Doshmangir L. Future of health care delivery in iran, opportunities and threats. Iran J Public Health 2013;42:23-30.
Bonu S, Gutierrez LC, Borghis A, Roche FC. Transformational trends confounding the South Asian health systems. Health Policy 2009;90:230-8.
Shaw C. How can hospital performance be measured and monitored?. World Health Organization Regional Office for Europe; 2003. p. 17.
Shahri S, Tabibi SJ, Nasiripour A, Ghaffari F. The Dimensions of health system performance evaluation with emphasis on the coverage of the vulnerable groups in Iran. Int J Med Res Health Sci 2016;5:9-17.
Ofori-Kuragu JK, Baiden BK, Badu E. Performance measurement tools for Ghanaian contractors. Int J Constr Manage 2016;16:13-26.
Rahimi H, Bahmaei J, Shojaei P, Kavosi Z, Khavasi M. Developing a strategy map to improve public hospitals performance with balanced scorecard and DEMATEL approach. Shiraz E-Med J 2018;19:e64056.
Walburg J, Bevan H, Wilderspin J. Performance Management in Health Care: Improving Patient Outcomes: An Integrated Approach. New York: Psychology Press; 2006.
Tawfik-Shukor AR, Klazinga NS, Arah OA. Comparing health system performance assessment and management approaches in the Netherlands and Ontario, Canada. BMC Health Serv Res 2007;7:25.
Kruk ME, Freedman LP. Assessing health system performance in developing countries: A review of the literature. Health Policy 2008;85:263-76.
Kelley E, Hurst J. Health care quality indicators project conceptual framework paper. Paris: OECD, 2006. Available from: http://www.oecd.org/health/health-systems/36262363.pdf [Last accessed 2019 Oct 23].
van den Berg MJ, Kringos DS, Marks LK, Klazinga NS. The Dutch health care performance report: Seven years of health care performance assessment in the Netherlands. Health Res Policy Syst 2014;12:1.
Handler A, Issel M, Turnock B. A conceptual framework to measure performance of the public health system. Am J Public Health 2001;91:1235-9.
Fekri O, Macarayan ER, Klazinga N. Health System Performance Assessment in the WHO European Region: Which Domains and Indicators have been used by Member States for its Measurement? Copenhagen: WHO Regional Office for Europe; 2018.
Kalinichenko O. Exploring performance assessment and incentive regulation in the context of primary health care. Portugal: Faculdade de Economia, Universidad do Algarve; 2017.
Carinci F, Van Gool K, Mainz J, Veillard J, Pichora EC, Januel JM, et al
. Towards actionable international comparisons of health system performance: Expert revision of the OECD framework and quality indicators. Int J Qual Health Care 2015;27:137-46.
NasrollahpourShirvani SD. A review of health systems performance assessment models. Clin Excellence 2016;5:98-116.
World Health Organization. Everybody's Business--Strengthening Health Systems to Improve Health Outcomes. WHO's Framework for Action. Geneva: World Health Organization; 2007.
Tashobya CK, da Silveira VC, Ssengooba F, Nabyonga-Orem J, Macq J, Criel B. Health systems performance assessment in low-income countries: Learning from international experiences. Global Health 2014;10:5.
Gu X, Itoh K. Performance indicators: Healthcare professionals' views. Int J Health Care Qual Assur 2016;29:801-15.
Veillard J, Guisset A-L, Garcia-Barbero M. Selection of Indicators for Hospital Performance Measurement: A Report on the 3rd
Workshop. Regional Office for Europe of the World Health Organization; 2003. p. 37.
Mehrtak M, Yusefzadeh H, Jaafaripooyan E. Pabon lasso and data envelopment analysis: A complementary approach to hospital performance measurement. Global J Health Sci 2014;6:107-16.
Tricco AC, Cardoso R, Thomas SM, Motiwala S, Sullivan S, Kealey MR, et al
. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: A scoping review. Implement Sci 2015;11:4.
Arksey H, O'Malley L. Scoping studies: Towards a methodological framework. Int J Social Res Methodol 2005;8:19-32.
Levac D, Colquhoun H, O'Brien KK. Scoping studies: Advancing the methodology. Implement Sci 2010;5:69.
Sheehan KJ, Sobolev B, Villán YFV, Guy P. Patient and system factors of time to surgery after hip fracture: A scoping review. BMJ Open 2017;7:e016939.
Silver MP, Hamilton AD, Biswas A, Warrick NI. A systematic review of physician retirement planning. Human Resour Health 2016;14:67.
Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs Health Sci 2013;15:398-405.
Terner M, D'Silva J, Tipper B, Krylova O, Webster G. Assessing primary healthcare using pan- Canadian indicators of health and health system performance. Healthc Q (Toronto, Ont) 2013;16:9-12.
Rohova M, Atanasova E, Dimova A, Koeva L, Koeva S. Health system performance assessment-an essential tool for health system improvement. J IMAB–Annual Proceeding Scientific Papers 2017;23:1778-83.
Mutale W, Balabanova D, Chintu N, Mwanamwenge MT, Ayles H. Application of system thinking concepts in health system strengthening in low-income settings: A proposed conceptual framework for the evaluation of a complex health system intervention: The case of the BHOMA intervention in Zambia. J Eval Clin Pract 2016;22:112-21.
Pallas SW, Curry L, Bashyal C, Berman P, Bradley EH. Improving health service delivery organisational performance in health systems: A taxonomy of strategy areas and conceptual framework for strategy selection. Int Health 2012;4:20-9.
Edward A, Kumar B, Kakar F, Salehi AS, Burnham G, Peters DH. Configuring balanced scorecards for measuring health system performance: Evidence from 5 years' evaluation in Afghanistan. PLoS Med 2011;8:e1001066.
World Health Organization. The world health report 2000: health systems: Improving performance. Geneva: World Health Organization; 2000.
Islam M. Health systems assessment approach: a how-to manual. USA, Arlington: Rational Pharmaceutical Management Plus; 2007. p. 374.
Makuta I, O'Hare B. Quality of governance, public spending on health and health status in Sub Saharan Africa: A panel data regression analysis. BMC Public Health 2015;15:932.
Lagomarsino G, Nachuk S, Kundra SS. Public stewardship of private providers in mixed health systems: Synthesis report from the Rockefeller Foundation-sponsored initiative on the role of the private sector in health systems in developing countries. Results for Development Institute, 2009.
Santos MA, Salomon VA, Marins FA. Analytic Network Process and balanced scorecard applied to the performance evaluation of public health systems. Pesquisa Operacional 2015;35:353-61.
Huselid MA. The impact of human resource management practices on turnover, productivity, and corporate financial performance. Acad Manage J 1995;38:635-72.
Forsberg E, Axelsson R, Arnetz B. Financial incentives in health care. The impact of performance-based reimbursement. Health Policy 2001;58:243-62.
Faraji Khiavi F, Maleki MR, Djafarian K, Vatan Khah S, Tabibi SJ. A comparative study on identifying core stages of policy-making in human resources for health: 2009, Jundishapur J Health Sci 2012;4:e93975.
Dehnavieh R, Haghdoost A, Khosravi A, Hoseinabadi F, Rahimi H, Poursheikhali A, et al
. The district health information system (DHIS2): A literature review and meta-synthesis of its strengths and operational challenges based on the experiences of 11 countries. Health Inf Manage J 2019;48:62-75.
Organization WH. Monitoring the building blocks of health systems: A handbook of indicators and their measurement strategies. Geneva, Switzerland: World Health Organization; 2010.
Health Finance & Governance Project. Health Systems Assessment Approach A How-To Manual. Version 3.0. Bethesda, MD: Health Finance & Governance Project, Abt Associates Inc.; 2017.
Sharifirad GR, Shamsi M, Pirzadeh A, Farzanegan PD. Quality gap in primary health care services in Isfahan: Women's perspective. J Educ Health Promot 2012;1:45.
Yakob B, Ncama BP. Measuring health system responsiveness at facility level in Ethiopia: Performance, correlates and implications. BMC Health Serv Res 2017;17:263.
Jahanmehr N, Rashidian A, Khosravi A, Farzadfar F, Shariati M, Majdzadeh R, et al
. A conceptual framework for evaluation of public health and primary care system performance in iran. Glob J Health Sci 2015;7:341-57.
Gauld R, Al-wahaibi S, Chisholm J, Crabbe R, Kwon B, Oh T, et al
. Scorecards for health system performance assessment: The New Zealand example. Health Policy 2011;103:200-8.
Kontodimopoulos N, Nanos P, Niakas D. Balancing efficiency of health services and equity of access in remote areas in Greece. Health Policy 2006;76:49-57.
Hollingsworth B. The measurement of efficiency and productivity of health care delivery. Health Econ 2008;17:1107-28.
Kringos DS, Boerma WGW, Hutchinson A, Van der Zee J, Groenewegen PP. The breadth of primary care: A systematic literature review of its core dimensions. BMC Health Serv Res 2010;10:65.
Marschall P, Flessa S. Efficiency of primary care in rural Burkina Faso. A two-stage DEA analysis. Health Econ Rev 2011;1:5.
Pelone F, Kringos DS, Romaniello A, Archibugi M, Salsiri C, Ricciardi W. Primary care efficiency measurement using data envelopment analysis: A systematic review. J Med Syst 2014;39:156.
Dehnavieh R, Noorihekmat S, Masoud A, Moghbeli MA, Rahimi H, Poursheikhali A, et al
. Evaluating the Tabriz health complex model, lessons to learn. Iran J Epidemiol 2018;13:59-70.
Pasarin MI, Berra S, Gonzalez A, Segura A, Tebé C, García-Altés A, et al
. Evaluation of primary care: The “Primary Care Assessment Tools - Facility version” for the Spanish health system. Gac Sanit 2013;27:12-8.
Khanassov V, Pluye P, Descoteaux S, Haggerty JL, Russell G, Gunn J, et al
. Organizational interventions improving access to community-based primary health care for vulnerable populations: A scoping review. Int J Equity Health 2016;15:168.
Lafortune C, Huson K, Santi S, Stolee P. Community-based primary health care for older adults: A qualitative study of the perceptions of clients, caregivers and health care providers. BMC Geriatr 2015;15:57.
Hofmarcher MM, Simon J, Peric N, Or Z, Smith P, Busse R. Indicators for structured monitoring of health system performance. Eur J Public Health 2016;26:149.
Tian M, Feng D, Chen X, Chen Y, Sun X, Xiang Y, et al
. China's rural public health system performance: A cross-sectional study. PLoS One 2013;8:e83822.
Sun D, Ahn H, Lievens T, Zeng W. Evaluation of the performance of national health systems in 2004-2011: An analysis of 173 countries. PLoS One 2017;12:e0173346.