Developing an Index to Measure Health System Performance: Measurement for Districts of Nepal

,

is vital to assess, compare and improve performance between sub-national levels, hence, there is a need to have a set of measurable and reliable indicators built up from a good information system.
There is a broad disparity in health outcomes for countries/states with similar levels of income and education. 3,4Some of these disparities are due to health system performance.Hence, meaningful, comparable information of health system performance can strengthen the foundations of health policy at various levels of health systems. 5e scope of using performance indicators are broad, ranging from examining national health system to individual patients at various levels of health system. 6ountries have been using various indicators to assess their health systems' performance looking at its dynamism.][9] The development of index is imperative to measure multidimensionality of health systems. 10The benefit of indices and positioning is that they provide policymakers and program managers various efficient decision-tools.The index can provide comprehensive measure of system performance.However, in practice, several existing composites are either opportunistic or incomplete or on questionable sources of data and these weaknesses can damage the credibility of composite. 10ny actions that intensely influence health, for instance educating public, are not part of health system, as the primary goal of education is not to improve health.In addition, it is critical to recognize that efforts to influence other sectors are clearly part of the health of health system when they improve determinants of health, such as educating public, developing infrastructures, or reducing social inequalities.These efforts at intersectoral action are deliberated to improve health.In addition, whatever the system or efforts, there are two goals 'responsiveness of the system and financial contribution', which are common to all.
The objective is not to compare among the countries or provinces or states.This could be useful to prioritize districts or provinces in health planning and programming.Aggregated information for the country may show good performance; however, at some districts or sub-national levels performance could be below par for the national figure.Therefore, the primary objective of this effort is to develop or identify an index, which can help or prioritize districts, province or countries to strengthen health system performance.

MEtHODOLOGIcAL APPrOAcH
2][13][14][15][16][17][18] These frameworks covers various ideas and approaches and are inclusive lists of numerous, and often overlapping, desirable attributes of health systems, including health related, inequalities in health, coverage, equitable financing, quality, consumer satisfaction, allocative and technical efficiency, cost containments, political acceptability, and financial sustainability.
We used indicators and often can serve as proxy to set of activities, whose primary goal is to maintain and improve health.These could be any activities, e.g.effort to develop infrastructures like water and sanitation, roads, supply system etc, and endeavour to increase literacy in the country, infrastructure development and development of capacity of health system in order to achieve health goals.
Experts will inevitably differ on weighing that is given to these components.Nevertheless, we believe that for the purposes of prioritizing activities, it could be useful to develop a consensus weighting function.This would allow composite measures for both goal attainment and efficiency to be calculated for health systems.Finally, the use of a composite measure of goal attainment will be limited but, like human development index, it can spark increased attention to the performance of health systems and the factors explaining such performance.The similar methodology for human development index has been used to deduce an index for health system performance. 19

selection of indicators and the rationale
Eleven health-related and socio-economic indicators, which are also included in the Millennium Development Goals, 20 country commitment, have been selected for this analysis:

Prevalence of underweight children under-five years of age (1-X 1 )
Child malnutrition is associated with socio-economic dimension of health like poverty, low levels of education, and poor access to health services.Malnourishment in children, even moderate, increases their risk of death, inhibits their cognitive development, and affects health status later in life.Adequate and good quality nutrition is the basis for development, health and survival of present and future generations.Good nutrition is equally important for women during pregnancy and lactation, which assist on children's key developmental paths, both physically and mentally.Circuitously, when there is optimal child growth for the majority of their people, then governments could be flourishing in their efforts to speed up economic development continuously.Hence, this provides the information on nutritional status of the country and the efforts of government on reducing malnutrition and under nutrition. 20oportion of 1 year-old children immunized against measles (X 2 ) Immunization is one of the vital elements for reducing under-five mortality.Governments in developing countries usually spend lot on immunization against measles and diphtheria, pertussis (whooping cough) and tetanus (DPT) as part of the basic health services.Among these vaccine-preventable diseases of childhood, measles is the leading cause of child mortality.The indicator provides a measure of the coverage and the quality of child health care system in the country in terms of availability, accessibility and coverage. 20,21oportion of births attended by skilled health personnel Every woman should have access to skilled care during pregnancy and at delivery to ensure detection and management of complications irrespective of their socioeconomic status.This is a process indicators of efforts of reducing maternal mortality, hence, this indicators is used as a proxy indicator.In addition, this provides the proxy information of availability of essential human resource capacity. 20,21ntraceptive prevalence rate (X 4 ) Contraceptive prevalence rate is an indicator of health, population, development and women's empowerment.It also provides as a proxy measure of access to reproductive health services, which has impact on maternal mortality, child mortality, gender empowerment, HIV/AIDS.The measure also signifies the extent of people's awareness level, accessibility and efforts to strength the health system. 20,22tenatal care coverage (X 5 ) Antenatal care coverage is an indicator of access and utilization of health services during pregnancy.The antenatal period is vital for the wellbeing of woman and their infants, as this provides the opportunity to reach pregnant women with interventions WHO recommends antenatal care at least four times, which increases the possibility of getting effective maternal health interventions.This provides information on utilization of reproductive health services and shows the availability and accessibility of the basic health services.The low rates shows the limitation on access, which might be due to unavailability of services, not promoted, or linked with high out of pocket expenditure.As this is one of the basic services and opted to reach out every woman of the societies, hence it can be used as proxy for overall accessibility of health services. 20,21verty incidence (1-X 6 ) The indicator allows for compare and aggregating progress across countries in reducing the number of people living under extreme poverty and for monitoring trends.Useful in resource allocation, prioritization and relates to inequality and social exclusion. 23Approximately 270 million people, mostly women and children have died due to poverty since 1990. 20,24This also serves as socio-economic dimension of health, service utilization and access of services and also provides information of government spending on development.

Proportion of population using an improved drinking
water source (X 7 )

Proportion of population using an improved sanitation facility (X 8 )
Good sanitation is important for urban and rural populations, but the risks are greater in urban areas where it is more difficult to avoid contact with waste.Use of an improved sanitation facility is a proxy for the use of basic sanitation and could serve as infrastructure development of the country. 20,21teracy rate of 15-24 year-olds (X 9 ) The youth literacy rate mirrors the outcomes of primary education at least for previous 10 years.As a measure of the effectiveness of the primary education system, it is often seen as a proxy measure of social progress and economic achievement.The literacy rate for this analysis is simply the complement of the illiteracy rate.It is not a measure of the quality and adequacy of the literacy level needed for individuals to function in a society.Indirectly, this indicator can provide the status of the population on accessing, seeking and accepting the health care services. 20

Postnatal care coverage(X 10 )
Postnatal care coverage is an indicator of access and utilization of health services during pregnancy.The postnatal period is vital for the wellbeing of woman and their infants; this provides information on utilization of reproductive health services and shows the availability and accessibility of the basic health services.The low rates shows the limitation on access, which might be due to unavailability of services, not promoted, or linked with high out of pocket expenditure.It also serves the purpose of measuring the accessibility, acceptability and availability of health services. 20,21

Proportion of maternal death as percentage of expected live birth (1-X 11 )
Complications during pregnancy and childbirth are a major cause of death and disability amongst women of reproductive age in developing countries.The maternal mortality ratio signifies the risk related with each pregnancy.Such deaths are affected by various factors, including general health status, education and services during pregnancy and childbirth.This is the impact indicator, which is used for developing index as most of them are outcome and process indicators. 20,21th a view of developing an index, indicators for the Nepal have been used to compare the districts' health systems performance.Using eleven critical health and socio-economic indicators, district level health system performance in Nepal could be quantified.The following are the sources of information from Nepal: Annual Report, DoHS 2006/ 2007; 25

Computing the index
The initial step is to form sub indices for each dimension.Minimum and maximum values (goalposts) need to be set so as to convert the indicators into indices between 0 and 1.Because the geometric mean used for aggregation, the maximum value does not affect the relative comparison (in percentage terms) between any districts or periods of time.The maximum values are set to the real observed maximum values of the indicators from the districts.The minimum values will affect comparison, so values that can be appropriately conceived of as subsistence values or 'natural' zeros are used.Progress is thus measure against minimum levels that a district need.

Aggregating the indices to produce an index
A simple average of the all the indices is computed.Sum of these indicators is divided by 11, as there are 11 indicators have been used (Table 1 -sheets for computing the district index values).The index value ranges from 0 (worst situation) and 1 (best situation).Higher the index better is the health system performance situation.

Validation of the computation
In order to recommend calculation of index for health system performance, computation is carried out by using data from all districts of Nepal for the given indicators.The index value generated has been compared with development index and socio-economic and infrastructure development index to validate.Despite there are various flaws in methodology and limitation of available data sources, this can be used for prioritizing health program activities rather than ranking districts for criticism.On top of that this calculation has been made as simple as possible so that any program managers and policy makers can calculate to prioritize activities.

Methodological Issues
Proxy indicators are used to measure health outcome either measuring health status like mortality with a view that health system activities could bring change in these status or measuring the process of care like utilization and availability. 29e responsiveness indicators are made by patients' acceptability and the efficiency indicators by fair access to the population on various services.The utilization of these health services are used as a proxy to acceptability and fair access e.g.immunization coverage, ANC and skilled birth attendance etc.The perplexity between performance indicators and health outcomes usually occurs and to shun this situation, health outcomes should relate to crude rates of adverse events in the population. 30her issues that have to be taken care in performance measurement is adjustment for case mix with outcome indicators since health care is only one determinants of health and that other factors, such as lifestyle environment, nutrition, poverty, literacy, and other could influence health outcomes. 31,32A fundamental benefit of process measures is that they are more sensitive than outcome measure to differences in the quality of care and is easy to interpret.Outcome measurement also reflects all aspects of the processes of services and not only those that are measurable or measured.

FINDINGs
Computation of each selected indicator of respective districts and their composite indices are referred (Table 1).Based on the computation, districts are ranked according to their index in an ascending order (Table 2).This computation is carried out as an illustration and for its validation, the indices are compared with available human development index, 33 and socio-economic and infrastructure development index. 34bles below have compared the findings with the human development index and socio-economic index and infrastructure development index with a view that development initiative directly or indirectly impact on health outcome.Table 3 compares human development index and health system performance index.We can clearly observe that districts are almost similar for both indices when it has been categorized in different levels.Health systems should be accountable for actions within organizations as well as key determinants of health, which are outside the scope of health system too.Through our measuring tool we manage to bring together not only health system but the indicators, which are outside of the scope of health system.However, there are some issues, which are debatable.
First, the scopes of availability of comparable data for these indicators are very much narrow, especially for the low -income countries.For instance, we have used available data from the Nepal from various years (within 2-3 years of time).However, data for system issues will not vary much in two to three years span of time and still can be used.
Secondly, some of the indicators have been used as a proxy for different pillars of health system development and its dimensions outside of its scope.For instance, proportion of birth attended by skill health personnel can be proxy for availability of the human resources in a country or district.Similarly, immunization coverage, contraceptive prevalence rate and others can be proxy to health service delivery.Proportion of provision of safe water supply and sanitation are the proxy for infrastructure development, where as the indicators for nutrition, education, poverty incidence provide information of social and economic determinants of health.
Thirdly, this index has not directly included fairness of finance and/or financial contribution towards development of health system.Critiques might consider this index incomprehensive because of non-inclusion of financial aspect of health systems development.However, the inclusion of indicators like poverty incidence and provision of safe water supply and sanitation are proxy to development initiative and spending, which indirectly reflect states or districts expenditures in health.Still in the interest of financial aspect of health system, the financial contribution can be used: e.g. the amount of spending on development sector, proportion distribution of health budget to sub-national levels and/or proportion of GDP spent on health for country level comparison.However, there are countries with lower GDP spending on health still having good indicators and vice versa. 38velopment of this index for measuring health system performance, we have tried to incorporate various aspects, which influence entire performance of health systems in different ways.First, it covers efforts and activities for the provision of basic and essential services.Secondly it has incorporated different indicators of non-health system determinants.Inclusion of these determinants also shows degree of intersectoral roles and contribution to performance of health systems and its outcome.Thirdly, this covers topics like coverage, responsiveness, outcome, accessibility, human resources capacity and health infrastructure development at national and sub-national levels.
Usually dimensions of performance include effectiveness, efficiency, quality of care, and equity, which are multifaceted and possess multiple definitions.Qualitative analysis with these dimensions cannot be left aside; however, the primary objective of developing this index is to provide a tool, which can help policy makers and program managers to prioritize the health programs and activities to those underprivileged areas, with a view that the certain population even doesn't have access to basic health services and infrastructures.Nevertheless, utilization of input, process and outcome measures by virtue reflects these dimensions in the index.
Provision of the index could be the basis for paradigm shift to pragmatic one from theoretical course of health policy.This index incorporated intrinsic goals of health system and its key measurable associates, it can lay foundation for a more pragmatic and scientific discourse on health policy and prioritization.This can serve as a convenient tool to monitor annual performance of health system for policy makers.Global institutionalization of this tool can assist donors and national and international organizations to prioritize their health development initiatives and agendas to various countries, who are more deprived.This can provide the snap shot for prioritizing the geographical location, which can help on in-depth analysis on gaps on strengthening health systems as well as other development initiatives.This index could be utilized for followings: sub-national monitoring of trend towards achievements related to the Millennium Development Goals; prioritization of resource allocation of the national health budget; harmonization of external development partner's assistance; to get an update of the overview of how many districts are required to build policy and program strategies to improve those indicators which contribute to low level of development; and an objective model to express health needs based on result-based approach is required.Findings of illustration of computing index from different dimensions of districts of Nepal have demonstrated similar finding with that of other universal indices used for development.Low performing districts for health system performance are also low in the context of human development index and socio-economic index and same applies to high performing districts.This evidence not only demonstrates the validation of this methodology of computing index for health system performance, but also provide evidence for policy makers to prioritize activities in order to strength.
Low performing districts like Kalikot, Humla, Dolpa, Rolpa, Mugu, Doti, Bajura etc are not only low performing in terms of health system performance index, but also remain similar to other health indicators, which are not used here for computing this index for districts.For instance, child mortality rate, disease morbidity and mortality rates, provision of hospital beds, availability of human resources and list goes on. 25Same applies to high performing districts for example Kathmandu, Lalitpur and Kaski.Kathmandu and Lalitpur are districts of Kathmandu valley, the capital of Nepal and Kaski has second biggest city of the country Pokhara, where all the facilities, resources and amenities are available.The development indices for these districts have also remained high. 33,34

WAY FOrWArDs
Different frameworks exist for health system performance measurement.Using indicators for set of health and non health related activities whose primary goal is to maintain and improve health can assist on developing index.Eleven health-related and socioeconomic indicators, which are also included in the Millennium Development Goals country commitment, have been selected to develop the index, which in turn demonstrated its comprehensiveness.This index can serve as a tool, which can be used on formulating health policy and prioritization of health programs to underprivileged areas.This index provides foundation for shifting theoretical course of health policy to empirical one.This can provide as a handy tool to monitor annual performance of health system for policy makers and health managers.This index could be utilized for followings: sub-national monitoring of trend towards achievements related to the Millennium Development Goals; prioritization of resource allocation of national health budget; harmonization of the external development partner's assistance; to get an update of the overview of how many districts are required to build policy and program strategies to improve those indicators which contribute to low level of development; and an objective model to express health needs based on resultbased approach is required.Global institutionalization of this tool can aid donors and national and international organizations to prioritize their health development initiative and agenda to various countries.
28all area estimation of poverty, caloric intake & malnutrition in Nepal, WFP/ WB/CBS, 2006; 26 National Management Information Project, Department of Water Supply & Sewerage, Annual Report 2008;27and VDC Profile of Nepal.28

Table 4
has compared the indices of socio-economic and infrastructural development and health system performance index.Similarly, when we categorized districts by high, intermediate and low indices, we can observe that categorization of districts for both indices are almost same.