Citation: Rudan I, Kapiriri L, Tomlinson M, Balliet M, Cohen B, et al. (2010) Evidence-Based Priority Setting for Health Care and Research: Tools to Support Policy in Maternal, Neonatal, and Child Health in Africa. PLoS Med 7(7): e1000308. doi:10.1371/journal.pmed.1000308
Published: July 13, 2010
Copyright: © 2010 Rudan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors received no specific funding for this article.
Competing interests: IR, LK, MT, and MC have all been involved in the development and implementation of the CHNRI methodology. IR and MC have been consultants of Child Health and Nutrition Research Initiative of the Global Forum of Health Research while developing the CHNRI methodology. The other coauthors have no competing interests to declare.
Abbreviations: CAM, Combined Approach Matrix; CHNRI, Child Health and Nutrition Research Initiative; CHOICE, Choosing Interventions that are Cost-Effective; COHRED, Council on Health Research for Development; DALY, disability-adjusted life year; DCPP, Disease Control Priorities Project; EHCP, Essential Health Care Package; ENHR, Essential National Health Research; LiST, Lives Saved Tool; MBB, Marginal Budgeting for Bottlenecks; WHO, World Health organization
Provenance: Commissioned; externally peer reviewed.
Priority Setting—Implicit or Explicit?
Priority setting is required in every health care system. It guides investments in health care and health research, and respects resource constraints. It happens continuously, with or without appropriate tools or processes. Although priority-setting decisions have been described as difficult, value laden, and political, only a few research groups are focused on advancing the theory of priority setting and the development and validation of priority setting tools –. These groups advocate the use of their tools, but their work is often not widely recognized, especially among the policy makers in developing countries, where these tools would be most helpful .
Our primary objective in this essay is to present the available tools for priority setting that could be used by policy makers in low-resource settings. We also provide an assessment of the applicability and strengths of different tools in the context of maternal and child health in sub-Saharan Africa.
The analyses of investments in neglected diseases showed that they lack transparent priority-setting processes . This persisting situation results in remarkable levels of inequity between investments in different health priorities –. Therefore, our secondary objective is to advocate for the use of the tools that could lead to more rational priority setting in sub-Saharan Africa. An optimal tool should be able to draw on the best local evidence and guide policy makers and governments to identify, prioritize, and implement evidence-based health interventions for scale-up and delivery.
Priority Setting in Low-Resource Settings—Mixed Evidence
Although there is currently insufficient evidence that the use of priority-setting tools improves health outcomes and reverses existing inequities, we have ample evidence that the lack of a rational and transparent process generates inequity and stagnation in mortality levels ,. Recently, Youngkong et al. conducted a systematic review of empirical studies on health care priority setting in low-income countries (Table 1) . The review found that policy makers in developing countries rarely consider using the available priority-setting tools, but also that the available tools lack credibility for priority setting in low-resource settings ,. This is mainly because it is not easy to validate the tools or to link their output with concrete follow-up actions and policy development . Indeed, it is difficult to prove beyond all doubt that investments in health care or health research are valuable to society when compared to alternative investments such as infrastructure or the economy.
Table 1. Priority setting exercises for health care or health research in low resource settings.doi:10.1371/journal.pmed.1000308.t001
However, there are many examples of countries that have reduced their maternal and child disease burden substantially from very high starting levels, and of others that keep failing to achieve progress . We also have strong evidence on the key determinants of those successes, which has been incorporated into various priority-setting tools ,–. The few studies that have evaluated processes in low-resource settings not using priority-setting tools found that most of them fell short on all four conditions of the “accountability for reasonableness” framework that assessed their basic legitimacy and fairness ,.
Moreover, there is evidence on the interventions and health research needed to improve maternal and child survival in low-resource settings. The key challenge is how to motivate and educate policy makers in sub-Saharan Africa to use the available priority-setting tools to direct the limited available resources into the most effective interventions and health research. We believe that addressing this challenge is critical, because it has been repeatedly shown that the scarcity of resources for health in sub-Saharan Africa is only part of the larger problem; the other part is that the scarce available resources are not being used efficiently by any standard, leading to tragic consequences for the population ,,.
Emerging Tools for Evidence-Based Priority Setting to Guide Health Care Policy
Several tools and processes are beginning to emerge as useful for priority setting in low-resource settings. In Table 1 we classify different methodologies by the context (national/global level) and scope (health care/health research prioritization). We also provide some essential information on the use of each method: (i) the setting; (ii) participants included in the process; (iii) the specific topic addressed; (iv) the criteria that were used for prioritization; (v) the process that was used; and (vi) the nature of the outcome. An in-depth comparative analysis of all these tools is beyond the scope of this essay, but in Table 1 we provide references to the key papers from which further information about those methods can be obtained (–; Lawn et al., manuscript in preparation).
Table 1 shows that the “burden of disease/cost effectiveness analysis,” promoted by the Disease Control Priorities Project (DCPP) , is an essential component of several tools that have been used for health care (interventions) prioritization: for example, the Marginal Budgeting for Bottlenecks (MBB) tool developed by UNICEF and The World Bank ; WHO-CHOICE (Choosing Interventions that are Cost-Effective) developed by the World Health Organization ,; and Lives Saved Tool (LiST) developed by Johns Hopkins University scientists and the Futures Institute . The DCPP approach for developing countries uses information on the burden of major diseases to assist decisions about the potential of affordable and effective interventions. The DCPP analysis identifies the “best buys,” i.e., the most cost-effective interventions in terms of DALYs saved per unit cost, that should compose a country's essential health care package (EHCP) . The EHCP should then influence program design and resource reallocation to help governments achieve the goal of reducing morbidity and mortality.
However, the DCPP authors note that factors other than cost-effectiveness influence priority setting in the real world, so the available evidence has to be considered in the context of local realities ,. Both MBB and WHO-CHOICE provide appropriate contextualization tools. However, the LiST software goes much further than any other tool in several dimensions. LiST contains an expansive evidence base of context-specific intervention effectiveness, generated by researchers from the WHO/UNICEF's Child Health Epidemiology Reference Group (CHERG) . It is a user-friendly decision-making computer software available in the public domain. It enables estimation of intervention impact on child mortality at national, regional, and global levels . Further important advantages of LiST include its validation in both African and South Asian contexts ,, an ability to perform very specific comparisons between alternative investment strategies over a specified time frame in terms of child survival outcomes –, its application of an equity lens , and easy translation of outcomes into program planning with convincing country-level examples .
Prioritizing Gaps in Health Research
Policy makers in low-resource settings also need to set priorities for health research. Table 1 shows that the CHNRI methodology has recently been used by several different groups to set health research priorities at the highest international level (–, Lawn et al., manuscript in preparation). However, there are several other tools for setting research priorities at the national level, which were reviewed and evaluated by Tomlinson et al. . Whereas CHNRI method had its first national-level implementation in South Africa only recently , other tools and processes have been dominant at the national level. The Council on Health Research for Development's approach (COHRED) has been implemented in Brazil, Cameroon, Peru, and Philippines; the Essential National Health Research (ENHR) approach in Cameroon and South Africa; and the Combined Approach Matrix (CAM) in Malaysia, Pakistan, and Argentina .
COHRED, ENHR, and CAM all were developed by committees set up by international agencies. All these methods are very specific about context, and they are excellent for organizing all the available information. However, they do little to provide an algorithm, based on a transparent set of criteria, that can distinguish among many competing research investment options ,. This does not, however, diminish their utility in most situations where the development of an evidence base is required. That phase can then be followed by Delphi-type consultation processes among a designated set of experts. For example, CAM does exceptionally well in addressing the two dimensions of the context that it finds the most important: the “public health” dimension and the “institutional” dimension. Having only two dimensions limits CAM's flexibility, though, and it is difficult to see how additional dimensions—e.g., uncertainty over the outcome (inherent to all health research); accounting for investment styles; accounting for the risk exposure and benefit potential of each research option; or the likelihood of obtaining funding support from donors—could be added . The same limitation is also true for COHRED and ENHR.
An emerging tool that is rapidly gaining popularity in the area of health research prioritization is the CHNRI methodology. It was developed over four years (2005–2008) with support from The World Bank for a transdisciplinary exercise of 15 experts. The experts assessed principles and practice of priority setting , reviewed universal challenges , developed a novel and robust conceptual framework , and provided guidance for stakeholder involvement  and for implementation of the method . Currently, they are in the process of developing user-friendly software that would enable simple, cheap, and effective conducting of CHNRI exercise via the internet.
The CHNRI methodology insists on transparency about the context in which priority setting takes place and the criteria used. It was initially developed for health research, but it has recently also been successfully used for health care and health interventions (Table 1) ,. Like the DCPP approach, it uses both cost-effectiveness and potential impact on disease burden as criteria. However, within a set of “standard” criteria, CHNRI also uses criteria relevant to the context—answerability, deliverability, affordability, sustainability, local capacity, likelihood of support, feasibility, equity, and others. The process is usually designed by policy makers or donors, conducted by technical experts in a transparent way (e.g., each vote counts equally), with a mechanism of stakeholder involvement. Stakeholders can assign different weights to the criteria used in the CHNRI exercise. The outcome is a comprehensive list with competing priorities ranked according to the combined scores they received in the process –. Such a list is helpful to policy makers because it provides an overview of strengths and weaknesses of competing investment options against many criteria, based on the collective input of technical experts. The list can also be adjusted by taking the values of many stakeholders into account.
The key challenges that need to be overcome in sub-Saharan Africa to improve the processes of prioritization in health care and health research include the following: increased acceptability and popularity with local policy makers, appreciation of the local context, clarity about the criteria used, transparency in the input from the stakeholders, and more specific guidance on translation into policy. Many papers that analyze the strategies for improving maternal and child survival conclude with highlighting the challenges such as integration, requirements for selection of community health workers, operational research into systems, among others. These are all admirable and important future areas of research. However, they are not exactly new, ground-breaking, or very specific, and the qualitative nature of the process frequently does not provide sufficient guidance to policy makers on the specific next steps. Tools such as LiST (for health care/interventions) and CHNRI (for health research) involve local experts and incorporate issues of local context into priority determination in a transparent, user-friendly, replicable, quantifiable and specific, algorithm-like manner. Both of these tools were primarily developed to address child health problems and should be considered by policy makers in the area of maternal and child health in sub-Saharan Africa.
The use of scientific evidence and principles in setting health priorities has an enormous potential to lead to more rational decision making, especially in low-resource settings where decision making has long lacked formal tools, processes, or an evidence base. We believe one cannot overstate the value of building and supporting the capacity of local experts and policy makers in sub-Saharan Africa to initiate and assist their own national government's policy formation process in maternal and child health, and of government's being able to generate rigorous credible “home grown” advice ,,. Regardless of the limitations of the available tools, we strongly recommend their use in development of sound maternal and child health policies in sub-Saharan Africa over the alternative of not using any method. The use of such tools would promote attention to objective evidence in public policy debates, often leading to decisions that are made are more clearly and in the public interest ,.
ICMJE criteria for authorship read and met: IR LK MT MB BC MC. Agree with the manuscript's results and conclusions: IR LK MT MB BC MC. Designed the experiments/the study: MT. Wrote the first draft of the paper: IR LK. Contributed to the writing of the paper: IR LK MT MB BC MC.
- 1. World Health Organization (2010) Report of the World Health Organization Expert Working Group on Research and Development Financing 2010. Available: http://www.who.int/phi/documents/RDFinancingwithISBN.pdf. Accessed 10 March 2010.
- 2. Kapiriri L, Norheim OF, Martin DK (2009) Fairness and accountability for reasonableness. Do the views of priority setting decision makers differ across health systems and levels of decision making? Soc Sci Med 68: 766–773.
- 3. Klein R (1998) Puzzling out priorities. Why we must acknowledge that rationing is a political process. Br Med J 317: 959–960.
- 4. Rudan I, Gibson J, Kapiriri L, Lansang MA, Hyder AA, et al. (2007) Setting priorities in global child health research investments: Assessment of principles and practice. Croat Med J 48: 595–604.
- 5. Moran M, Guzman J, Ropars A-L, McDonald A, Jameson N, Omune B, Ryan S, Wu L (2009) Neglected disease research and development: how much are we really spending? PLoS Med 6: e30. doi:10.1371/journal.pmed.1000030.
- 6. Enserink M (2009) Some neglected diseases are more neglected than others. Science 323: 700.
- 7. Youngkong S, Kapiriri L, Baltussen R (2009) Setting priorities for health interventions in developing countries: a review of empirical studies. Trop Med Int Health 14: 930–939.
- 8. Kapiriri L, Arnesen T, Norheim OF (2004) Is cost-effectiveness analysis preferred to severity of disease as the main guiding principle in priority setting in resource poor settings? The case of Uganda. Cost Effect Resource Allocat 2: 1–11.
- 9. Allen L (2010) The art of evaluating the impact of medical science. Bull WHO 88: 4.
- 10. Rudan I, Chan KY, Zhang JSF, Theodoratou E, Feng XL, Salomon J, Lawn JE, Cousens S, Black RE, Guo Y, Campbell H (2010) Causes of deaths in children younger than 5 years in China in 2008. Lancet 375: 1083–1089.
- 11. Kapiriri L, Martin DK (2006) Priority setting in developing countries health care institutions: the case of a Ugandan hospital. BMC Health Serv Res 6: 127.
- 12. Mshana S, Shemilu H, Ndawi B (2007) What do district health planners in Tanzania think about improving priority setting using ‘Accountability for Reasonableness’? BMC Health Serv Res 7: 180.
- 13. Laxminarayan R, Mills AJ, Breman JG, Measham AR, Alleyne G, et al. (2006) Advancement of global health: key messages from the Disease Control Priorities Project. Lancet 367: 1193.
- 14. UNICEF/The World Bank (2010) Marginal Budgeting for Bottlenecks. Available: http://www.who.int/pmnch/topics/economics/costingtools_resources/en/index.html. Accessed: 10 March 2010.
- 15. Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, et al. (2005) Cost effectiveness analysis of strategies for maternal and neonatal health in developing countries. BMJ 331: 1107.
- 16. Victora CG (2010) LiST: using epidemiology to guide child survival policymaking and programming. Int J Epidemiol 39: (Suppl 1)i1–2.
- 17. Bobadilla JL, Cowley P, Musgrove P, Saxenian H (1992) Design, content and financing of an essential national package of health services. Bull World Health Organ 72: 653–662.
- 18. Rudan I, Chopra M, Kapiriri L, Gibson J, Lansang MA, et al. (2008) Setting priorities in global child health research investments: universal challenges and conceptual framework. Croat Med J 49: 307–317.
- 19. Kapiriri L, Tomlinson M, Gibson J, Chopra M, El Arifeen S, et al. (2007) Setting priorities in global child health research investments: Addressing the values of the stakeholders. Croat Med J 48: 618–627.
- 20. Rudan I, Gibson JL, Ameratunga S, El Arifeen S, Bhutta ZA, et al. (2008) Setting priorities in global child health research investments: Guidelines for implementation of the CHNRI method. Croat Med J 49: 720–733.
- 21. Walley J, Lawn JE, Tinker A, De Francisco A, Chopra M, et al. (2008) Primary Health Care: making Alma Ata a reality. Lancet 372: 1001–1007.
- 22. Pattinson R, Lawn JE, Darmstadt G, Rubens C, Flenady V, et al. (2010) Priority interventions to reduce the global burden of stillbirths. Lancet. In press.
- 23. Tomlinson M, Rudan I, Saxena S, Swartz L, Tsai AC, Patel V (2009) Setting investment priorities for research in global mental health. Bull World Health Organ 87: 438–446.
- 24. Costello A, Filippi V, Kubba T, Horton R (2007) Research challenges to improve maternal and child survival. Lancet 369: 1240–1243.
- 25. Bahl R, Martines J, Ali N, Bhan MK, Carlo W, et al. (2009) Research priorities to reduce global mortality from newborn infections by 2015. Pediatr Inf Dis J 28: (Suppl 1)S43–S48.
- 26. Fontaine O, Kosek M, Bhatnagar S, Boschi-Pinto C, Chan KY, et al. (2009) Setting research priorities to reduce global mortality from childhood diarrhoea by 2015. PLoS Med 6: e41. doi:10.1371/journal.pmed.1000041.
- 27. Rudan I, El Arifeen S, Black RE, Campbell H (2007) Childhood pneumonia and diarrhoea: Setting our priorities right. Lancet Inf Dis 7: 56–61.
- 28. Brown KH, Hess SY, Boy E, Gibson RS, Horton S, et al. (2009) Setting priorities for zinc-related health research to reduce children's disease burden worldwide: An application of the Child Health and Nutrition Research Initiative's research priority-setting method. Public Health Nutr 12: 389–396.
- 29. Tomlinson M, Swartz L, Officer A, Chan KY, Rudan I, Saxena S (2009) Research priorities for health of people with disabilities: an expert opinion exercise. Lancet 374: 1857–1862.
- 30. Tomlinson M, Chopra M, Hoosein N, Rudan I (2010) Research priority setting processes at national level: Good practices for health research. Health Syst Res Policy. In press.
- 31. Tomlinson M, Chopra M, Sanders D, Bradshaw D, Hendricks M, et al. (2007) Setting priorities in child health research investments for South Africa. PLoS Med 4: e259. doi:10.1371/journal.pmed.0040259.
- 32. Rudan I (2009) The complex challenge of setting priorities in health research investments. Indian J Med Res 129: 351–353.
- 33. Walker N, Fischer-Walker C, Bryce J, Bahl R, Cousens S, CHERG Review Groups on Intervention Effects (2010) Standards for CHERG reviews of intervention effects on child survival. Int J Epidemiol 39: (Suppl 1)i21–31.
- 34. Friberg IK, Bhutta ZA, Darmstadt GL, Bang A, Cousens S, et al. (2010) Comparing modelled predictions of neonatal mortality impacts using LiST with observed results of community-based intervention trials in South Asia. Int J Epidemiol 39: (Suppl 1)i11–20.
- 35. Hazel E, Gilroy K, Friberg I, Black RE, Bryce J, et al. (2010) Comparing modelled to measured mortality reductions: applying the Lives Saved Tool to evaluation data from the Accelerated Child Survival Programme in West Africa. Int J Epidemiol 39: (Suppl 1)i32–9.
- 36. Amouzou A, Richard SA, Friberg IK, Bryce J, Baqui AH, et al. (2010) How well does LiST capture mortality by wealth quintile? A comparison of measured versus modelled mortality rates among children under-five in Bangladesh. Int J Epidemiol 39: (Suppl 1)i186–92.
- 37. Bryce J, Friberg IK, Kraushaar D, Nsona H, Afenyadu GY, et al. (2010) LiST as a catalyst in program planning: experiences from Burkina Faso, Ghana and Malawi. Int J Epidemiol 39: (Suppl 1)i40–7.