Background

During the last three decades, significant achievements have been made in improving the health of the world population [1, 2]. This can be attributed in part to the Millennium Development Goals [1] and is further augmented by the launch of the Sustainable Development Goals in September 2015 [2]. In developing countries, however, still more progress needs to be made. In these countries, still far too many women die during childbirth [1,2,3,4,5], too many children die from preventable causes [1, 2, 4, 6] and too many adults die from treatable infectious and non-communicable diseases [6, 7]. Reducing mortality and morbidity is the main focus of primary healthcare [1, 2]. While progress has been made in communicable diseases, the burden of non-communicable diseases is straining developing countries’ healthcare resources [6, 7].

Primary healthcare is an essential and critical type of healthcare delivery that addresses the health needs of the population usually delivered at centres called Primary Health Centres (PHC) [8]. In the last decade, the achievements of healthcare have gained significant attention as the ‘performance of healthcare’ which is not specific to PHC [9,10,11,12,13,14]. In order to monitor their health system, countries carry out performance assessment; the performance is fulfilling one’s obligation, in a way that releases one from liabilities [15]. Performance assessment can be defined as a ‘coherent evaluation system which assesses the whole occupational functioning including its constituent parts’ [16]. A comprehensive assessment of the system in developing countries is vital for determining the gap between demand for services and the ability of the healthcare systems to reciprocate.

In this regard, the World Health Organization’s (WHO) health systems performance assessment framework serves as the global benchmark. According to the framework, health systems performance objectives are good health, responsiveness and fair financial contribution [17]. The framework describes six aspects of performance assessment: overall level of health considering the general health of population; distribution of health in population (healthcare services coverage); overall level of responsiveness indicated by the quality of care, satisfaction of care and availability of services; distribution of responsiveness; distribution of resources such as human resources and care facilities; distribution of financial contribution from various agents and their optimal use [17]. Even though it would be ideal, many developing countries might lack adequate resources and data to assess PHC performance according to this framework, and hence, may have reservations about applying the framework [10, 18].

The specific measures of PHC performance assessment which are used in developing countries are not well known. In the most recent review of health care performance measurements, no distinction was made between developed and developing countries [19]. The Primary Healthcare Systems are evolving at a faster pace in developing countries than before and lacks clarity, hence the Primary Health Care Performance Initiative calls the system’s performance a “Black Box” and identifies an urgent need to build on the existing knowledge [20].

The aim of this study is to review the scientific literature on measures of PHC performance assessment used in developing countries and to compare them with the WHO framework for health care performance assessment for comprehensiveness. Based on the literature, four aspects of healthcare performance measurement [9, 11, 21] are emphasized in this review:

  1. 1.

    The methods for assessing performance in health care [22]

  2. 2.

    The quality of measurement (validity and reliability) [16]

  3. 3.

    The professional actions, that is, the performance of the provider [23,24,25]

  4. 4.

    The levels covered by the measurement: the level of the patient, that of the community, district/state level and/or country level. Concurrently, measures could capture different perspectives such as that of the health care provider or that of the patient [26].

Methods

This paper employs a review of the literature on PHC performance assessment in developing countries. The flow diagram of the selection and search process is depicted in Fig. 1.

Fig. 1
figure 1

Methodology of Review

Full size image

Search strategy

An initial search for articles from the following databases was carried out: Hospital/Health care (Pubmed, Embase), Psychology (JSTOR, PsycINFO) and Business (Emerald Insight). Endnote7 software was utilised to download and select articles. As the aim of the review was to include articles on PHC performance assessment in developing countries that are published in English, the search terms included ‘primary health centre’ or ‘health centre’ and ‘performance assessment’ or ‘performance’ and ‘developing country’ {(primary health centre or health centre) and (performance assessment or performance) and developing country}. The period of publication for the search was from the year 1979, a year after Alma Ata up to December 2014. Further, an additional search was done from January 2015 to October 2016 to include recent articles. The search terms were tested and Mesh terms were employed during search when available in the database. The search results from the databases identified were combined to eliminate duplicates. The details of the search strategy are given in Additional file 1: Appendices 1 and 2. A similar search was conducted in Google scholar and the articles fulfilling inclusion criteria were included for full review (Fig. 1).

The inclusion criteria employed were:

  1. a)

    Empirical articles with measurement of PHC or professional performance (from professional or health centre perspectives);

  2. b)

    The measurements were conducted in a developing country; and

  3. c)

    Papers were published in English

Articles that reported only on methodology of research, discussion on performance/performance assessment and review of literature were excluded. The authenticity of this review was ensured by including only published empirical articles, avoiding grey literature and reviews. On the basis of these inclusion and exclusion criteria, the articles were screened first by title, then by the abstract and full article, to select the relevant articles for full paper review.

Quality assessment

Next, the selected full articles were assessed for quality. The complete evaluation of methodological characteristics as suggested by COSMIN and Noben and colleagues could not be applied, as methodologies of the articles selected were not advanced enough [27, 28]. Hence, a simpler instrument was developed, based on the quality of the description and the quality of the measurement property (the reliability and validity) itself. It is theoretically possible that the measurement property is weak although the description is good [28]. With our instrument for assessing the quality of description, the articles were scored as:0, if there was no description of the measurement property;1, if only a few points of the measurement property were described (little information); and 2, if there was a detailed description of the measure in the article (good information).

The quality of the measurement property was assessed based on four criteria:

  1. 1.

    The standardization: method of data collection was noted and the standardization by pretesting of the tool and/or the training of researchers was reported.

  2. 2.

    The triangulation of the data collection method and/or data source with significantly similar results.

  3. 3.

    The measure used was well established/widely published, this was done by verifying references and other cited publications.

  4. 4.

    The reliability and validity of the measures was tested using statistical methods and compared with a global standard (WHO performance assessment framework) [29].

The quality was scored as follows:

  • 0 if none of the above criteria was fulfilled

  • 1 if only standardization of the measure and/or researchers by training was done

  • 2 if triangulation of methods/data sources was done

  • 3 if the methods used for analysis were well established/published

  • 4 if validity of the measure was tested with good results

The results were relegated to a lower number if the higher criteria were not fully met or exclusively mentioned in the article. The qualitative article was scored between 0 and 2 as its measures would be unique, based on the theoretical framework used (scores 3 and 4 were irrelevant). The papers were analysed for quality by the first two authors independently, the differences were discussed and agreed upon.

Descriptive analysis

The selected full articles were analysed and reviewed to understand and describe how the performance of PHCs had been assessed in developing countries. All selected articles were studied to understand how they had defined performance and the perspectives of the performance assessment (provider or patient). The topographical and content analysis of measures was done and these were compared with the measures of WHO performance assessment framework for comprehensiveness [29].

Ethics

No ethical approval was needed for this narrative review.

Results

Literature search results

After downloading the articles (4359 articles) using the search strategy described above, duplicates were removed, inclusion and exclusion criteria were applied systematically, which resulted in the selection of eight articles from the initial search and three articles from the additional search for the review. Further, searches from the reference tracking and in Google Scholar resulted in another four articles (Fig. 1). Thus, in total fifteen articles were included for quality assessment and content analysis (Table 1). Table 1 provides an overview of these fifteen articles.

Table 1 Description of articles
Full size table

Quality of measures

Fourteen articles had a good description of the measurement properties. The quality of the measures varied. Measures in eleven articles scored 3, two articles scored 2 and two articles scored 1 (Table 1). None of the measures was tested for validity with good results.

Description of the PHC performance measures

Empirical research on PHC performance measures was published from Brazil, India, Papua New Guinea, Egypt, Bangladesh, Armenia, Malawi, Tanzania, Rwanda, Serbia and other developing countries. A descriptive analysis revealed that PHC performance was distinguished as individual staff/personnel performance and that of the centre/health system performance with a focus on consumer’s and/or provider’s perspectives. Figure 2 depicts the perspective and level of assessment.

Fig. 2
figure 2

Overview of Primary Healthcare Performance Assessment in Developing Countries

Full size image

Method of performance assessment

Nine articles included in this review used quantitative methods, one article had exclusively employed qualitative methods and five had employed mixed methods. The methods of data collection included were interviews with clients, health workers, key informants and/or stakeholders [30,31,32,33,34,35,36,37,38], focus group discussions [30], direct observation [35, 39], facility-based survey [34, 35, 40, 41], and secondary data analysis (records on costs, infrastructure, service provision, health indicators, number of patients served and other patient details) [30, 31, 34, 39, 42].

Performance of the professionals

Five studies evaluated the personnel performance assessment (nurses, midwives, management team, community health workers and medical professionals). The assessment considered competency/clinical services and non-competency/non-clinical service-based components. The personnel performance was assessed using the Quick Investigation of Quality tool [34] (Fort and Voltero 2004). Adherence to the protocol was the most common method for competency based personnel performance assessment [30, 33, 35, 36, 38]. For the non-competency based personnel performance assessment, the measures used were punctuality, response time, absenteeism, time of reporting to work, time spent with patients, waiting time and cleanliness [33, 36]. Reproductive healthcare delivery was evaluated in all the papers [30, 33, 35, 36, 38].

It was found that autonomy of the team, availability of manpower, clarity in the job description, roles and responsibilities, working conditions, workload and environment, the level of motivation, their education and training along with good supervisory practices contributed to the performance of healthcare personnel [33, 38].

Performance of the health system/Centre

Of the 15 articles selected, 9 studies assessed the performance of the centre or the health system [31, 32, 34, 36, 37, 39,40,41,42,43,44]. Satisfaction of patients, community, care providers and other stakeholders were the most common measures utilized for centre/health system performance assessment. The performance of the health system was assessed from the provider’s perspective (4 articles) [31, 34, 37, 39], the consumer’s perspective (4 articles) [32, 40, 42, 43] and both perspectives (2 article) [41, 44] at various levels.

Perspective and structural levels of performance

The target groups in the studies were individual patients, local community, district or country level. Health care personnel such as nurses, doctors and health managers along with other key stakeholders like local governing bodies were included. From the respondent’s perspective, performance was defined as satisfaction, accessibility of the centre, case detection and success of the treatment provided in the country, care experiences, establishment of patient’s bonds for treatment success and organizational care. Centre performance assessment was assessed as provider and patient bonding leading to therapy compliance and treatment success. This was also assessed in terms of costs and effectiveness of the services provided, the satisfaction of the providers and other stakeholders on the availability of resources and functioning of the centre. The scope of the centre performance assessment was at the structural levels of centre, community, district or country.

Comparison with the WHO aspects of health systems performance assessment

The measures described in the articles were compared with the six aspects of the WHO health systems performance assessment framework (Table 2) [29]. In the articles included, measures for three aspects (overall level of responsiveness, distribution of responsiveness and distribution of resources) were considered in relation to specific diseases/services, though the WHO framework uses system-specific measures. Measures for disease/service specific WHO aspects (overall level of health, distribution of health in population and distribution of financial contribution) were less well represented in the fifteen articles.

Table 2 Measures categorized as per WHO aspects of health systems performance assessment
Full size table

Discussion

The aim of this review was to describe the measures of PHC performance assessment used in developing countries as published in the empirical literature and to compare them with the WHO framework for health systems performance assessment. The fifteen articles that were considered provided scarce information on measurement quality and covered limited aspects of PHC performance when compared to the WHO health systems performance assessment framework. Measures were found both at the level of the performance of professionals and of the centre, and further measures that addressed the satisfaction of the performance by stakeholders were found. These measures will be discussed below.

Performance of professionals

Personnel performance was based on observation methods, assessed competency and non-competency based tasks. This correlated highly with patient satisfaction, an indicator of centre performance, as well as that of availability of resources, support and culture of the organization [45]. The investigators used the ‘Quick Investigation of Quality’ tool that had been validated earlier [45, 46]. Such validated tools for observation of professional performance provide a quick and effective method for assessment of PHC personnel performance in a developing country. As reproductive and child health is one of the main focus areas of primary healthcare and an important health indicator, it is hence appropriate to use this in the evaluation of PHC performance as well.

Performance of the Centre

The performance of the centre was assessed using the Primary Care Assessment Tool [40, 41, 43]. This tool was validated in a developing country for family healthcare [41, 47]. It is interesting to note that this instrument could be used to assess the performance from the consumer’s as well as from the provider’s perspective. This tool provides an extensive list of surveys on different attributes of PHC. Based on the requirement of the evaluation, one could consider specific components. In other studies, measures such as costs, patients served and effectiveness along with the client satisfaction were identified. Cost effectiveness is an appropriate measure that can be used across all PHCs [48] and thus, helps in making right choices by identifying the most effective service or intervention or centre [48].

Satisfaction with performance by various stakeholders

Satisfaction of various stakeholders including patients and providers is an accepted standard in performance assessment as they are interrelated. Community participation in the functioning of the PHC highly influences the performance of PHC and its personnel, so it could be included as a component in the PHC performance assessment [34].

WHO aspects of performance assessment

The current practices of PHC performance assessment in developing countries were analysed with the WHO framework for performance assessment [17, 29]. The WHO framework is completely focused on output and outcome, with the structure and process considered intrinsic to the system. However, for ease and clarity, many authors have used the Donabedian model for assessing performance in primary health care [19, 23, 26, 49]. The current review highlights the limited representation of the performance measures in relation to aspects of the WHO framework (Table 2). In the articles, even the tools used to assess personnel performance (Quick Investigation of Quality tool) and centre performance (Primary Care Assessment Tool) did not represent measures from three aspects: overall level of health, distribution of health in the population and the distribution of the financial contribution [40, 41, 47].

Due to lack of resources and data [10, 18], covering the complete list of WHO aspects to the full extent will be difficult for developing countries. However, a concise list of measures with an appropriate representation of six WHO aspects and requiring minimum resources and data, needs to be developed for PHC performance assessment in developing countries. If the assessment uses standardised measures useful comparisons across regions/countries would be possible. Since the search results indicated that a very small number of articles were published, further research needs to be conducted in the developing countries on PHC performance assessment, enabling cross learning and knowledge base enhancement [21].

Discussion on methodological quality of articles

Although the number of articles in this review was limited, the studies covered diverse countries and continents. Assessing the quality of the papers was a challenge as articles used qualitative, quantitative and mixed methodology. The articles demonstrated reliability and validity methods such as correlating provider and user experiences, data/method triangulation, standardised tools/training of researchers. Hence, there is a need for research in developing countries to establish quality of measurement and standardisation of PHC performance assessment using rigorous statistical methods.

Implications for future research

PHC performance assessment in developing countries is an emerging field but it is fragmented at present. Though human resource for health is a component of the WHO framework, the various sub-components to be included are not clear. Evaluation of the PHCs is an on-going exercise, but yet there is no established standard for assessing performance. Studies have shown that the performance of PHC depends on several things, one critical factor being personnel performance [21, 29, 50,51,52].

Implications for practice

What is measured, can be controlled. However, not all aspects of PHC performance were yet covered. Measures such as skill sets for execution of healthcare focusing on mother and child health, are already well available. They were emphasised in five studies of our review and this is in line with Millennium Development Goals and Sustainable Development Goals [1, 2]. However, other aspects like coverage/distribution of health, overall health of the population, distribution of financial contribution, including program specific achievements, various service components and process of primary care delivery are still not covered. The performance measures should be appropriate and adequate enough to enable accurate assessment on an ongoing basis to aid in monitoring and efficient management of the personnel/system. There is urgency to develop new and additional measures for PHC performance in developing countries. PHC performance is a matter of immense importance for policy agenda and political priorities [53].

Conclusions

Although developing countries may have difficulties in applying the entire WHO framework, the current measures for assessment of PHC performance published in scientific journals are limited in scope and lack validation. The standard health indicators for the overall level of health and for the distribution of health in the population were represented least. A comprehensive assessment of primary healthcare can be achieved by integrating personnel performance with that of centre performance. Representation/inputs from both sides of the service delivery, the management and the consumer/public, that is, including the provider’s perspective and consumer’s perspective are vital. From this review, it can be concluded that existing measures for PHC performance assessment in developing countries need to be validated and concise measures for neglected aspects need to be developed.