Advertisement
Policy Forum

Building the Field of Health Policy and Systems Research: Social Science Matters

  • Lucy Gilson mail,

    lucy.gilson@uct.ac.za

    Affiliations: School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa, Department of Global Health and Development, London School of Hygiene and Tropical Medicine, London, United Kingdom

    X
  • Kara Hanson,

    Affiliation: Department of Global Health and Development, London School of Hygiene and Tropical Medicine, London, United Kingdom

    X
  • Kabir Sheikh,

    Affiliation: Public Health Foundation of India, New Delhi, India

    X
  • Irene Akua Agyepong,

    Affiliation: Ghana Health Service/School of Public Health, University of Ghana, Accra, Ghana

    X
  • Freddie Ssengooba,

    Affiliation: School of Public Health, Makerere University, Kampala, Uganda

    X
  • Sara Bennett

    Affiliation: Health Systems Programme, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America

    X
  • Published: August 23, 2011
  • DOI: 10.1371/journal.pmed.1001079
  • Published in PLOS Medicine

PLoS Medicine Series on HPSR

Following the First Global Symposium on Health Systems Research in Montreux in November 2010, PLoS Medicine commissioned three articles on the state-of-the-art in Health Policy and Systems Research (HPSR). Three Policy Forum articles, authored by a diverse group of global health academics, critically examine the current challenges to the field and lay out what is needed to build capacity in HPSR and support local policy development and health systems strengthening, especially in low- and middle-income countries.

Paper 1. Kabir Sheikh and colleagues. Building the Field of Health Policy and Systems Research: Framing the Questions.

Paper 2. Lucy Gilson and colleagues. Building the Field of Health Policy and Systems Research: Social Science Matters.

Paper 3. Sara Bennett and colleagues. Building the Field of Health Policy and Systems Research: An Agenda for Action.

Summary Points

  • All researchers hold a knowledge paradigm that frames their understanding of reality and of the functions and nature of research. Some disciplines are dominated by a particular paradigm and some are spread across paradigms.
  • The criticisms that Health Policy and Systems Research (HPSR) is too context specific, does not offer clear lessons for policy makers, and is not rigorous are partly a reflection of differences in knowledge paradigms between those with predominantly clinical, biomedical, and epidemiological backgrounds, underpinned by a positivist paradigm, and those with social science backgrounds underpinned by a relativist paradigm.
  • Health policies and systems are complex social and political phenomena, constructed by human action rather than naturally occurring. Relativist social science perspectives are, therefore, of particular relevance to HPSR as they recognise that all phenomena are in essence constructed through human behaviour and interpretation.
  • Social science insights that can advance the science of HPSR include approaches to generalising from rich understanding of context; supporting policy learning; and enhancing research rigour and quality.

Introduction

The first paper in this series on building the field of Health Policy and Systems Research (HPSR) in low- and middle-income countries (LMICs) [1] outlined the scope and questions of the field and highlighted the key challenges and opportunities it is currently facing. This paper examines more closely one key challenge, the risk of disciplinary capture—the imposition of a particular knowledge frame on the field, privileging some questions and methodologies above others. In HPSR the risk of disciplinary capture can be seen in the current methodological critique of the field, with consequences for its status and development (especially when expressed by research leaders).

The main criticisms are reported to be: that the context specificity of the research makes generalisation from its findings difficult; lack of sufficiently clear conclusions for policy makers; and questionable quality and rigour [2]. Some critique is certainly warranted and has come from HPS researchers themselves. However, this critique also reflects a clash of knowledge paradigms, between some of those with clinical, biomedical, and epidemiological backgrounds and those with social science backgrounds. Yet, as HPSR is defined by the topics and questions it considers rather than a particular disciplinary approach, it requires engagement across disciplines; indeed, understanding the complexity of health policy and systems demands multi- and inter-disciplinary inquiry [3].

To develop the science of HPSR it is, therefore, important to start by recognising the diversity of disciplinary perspectives, as well as shared concerns. Richer methodologies for addressing these concerns must then be developed. And, as health policies and systems are themselves social and political constructions, it is important to acknowledge the particular value of social science perspectives in the field. Each of these issues is addressed in the following sections, and they are considered further in paper three of the series [4].

Knowledge Paradigms

Figure 1 characterises key areas of difference between the dominant knowledge paradigms that underpin the disciplines applied within HPSR. The figure deliberately polarises the paradigms to spark debate. Some disciplines are dominated by a particular paradigm and some are spread across paradigms.

thumbnail

Figure 1. Core differences between knowledge paradigms.

doi:10.1371/journal.pmed.1001079.g001

The positivist worldview is reflected in much clinical, biomedical, and epidemiological, and some social science, research. This view starts from the same position as the natural and physical sciences. The phenomena being investigated comprise a set of facts, a single reality that can be observed and measured by the researcher without disturbing them. The central aim of research is to detect causal mechanisms through the deductive process of testing hypotheses derived from theory and past experience against empirical facts. At their simplest, such mechanisms represent the prediction that “x will cause y” in any other setting. Simple HPSR hypotheses might include, for example, “limited financial incentives cause low motivation” or “a lack of health facilities undermines access to health services.” Sometimes such hypotheses are tested through statistical analysis of secondary data [5]; sometimes studies are designed to allow hypotheses to be tested [6]. Indeed, the positivist perspective underpins the recent rise of experimental methodology in impact evaluation. As the emphasis in such studies is on measuring the magnitude of an intervention's impact, and ensuring that this estimate is unbiased, careful attention is paid to selecting an appropriate control group (randomized or otherwise) and controlling the influence of possible confounding factors. Much less emphasis is placed on understanding how the intervention works and which contextual or other factors mediate its impact.

Much social science work that is qualitative is located at the relativist end of the spectrum. Such research is essentially based on the understanding that the world around us is subject to human interpretation. Health policies and systems are, therefore, understood to be constructed and brought alive by social actors through the meaning they attach to (their interpretations of) their experiences. Whereas positivist researchers focus on facts and regularities (that is, causes and effects), relativist researchers see interpretations as the primary subject of inquiry, proposing that different interpretations of the same experience represent multiple realities. In this tradition, researchers study human behaviour in everyday or natural settings, generating qualitative data that are primarily analysed inductively to generate categories and explanations of experience. Such analysis also involves interpretation by the researcher, in interaction with respondents. It may be guided by, and/or generate, what is called middle range theory, i.e., ideas about how the world works, comprising categories and concepts derived from analysis, and suggestions about how they are linked together. Middle range theory may be tested against evidence through the process of analysis or highlights questions and ideas to be considered in future studies.

Relativist HPSR studies focus, for example, on how health system actors understand and experience particular services or policies [7], and what social and political processes, including power relations, influence them [8],[9]. The development and testing of middle range theory is also supported by studies that adopt a critical realist position. This knowledge paradigm falls somewhere in the spectrum between positivism and relativism, and is of growing interest in HPSR [10] (see FEMhealth, http://www.abdn.ac.uk/femhealth/). However, these sorts of questions are still only quite rarely addressed in the wider HPSR literature [1].

Shared Concerns and the Value of Multiple Perspectives

Although HPS researchers from different disciplinary traditions have some difficulty understanding each other's perspectives, they also have some shared starting points: a common focus, health policies and systems, and a concern about how to strengthen health systems to benefit those being served by them. The complexity of the phenomena being investigated may also generate a willingness to think creatively about how to investigate issues. Therefore, HPS researchers tend not to fall at the extreme ends of the spectrum outlined in Figure 1—and this makes multi- and inter-disciplinary work more possible.

Review of existing HPSR work demonstrates, moreover, that bringing together research from different traditions generates broader and deeper understanding on the issues of focus. Box 1, for example, shows the breadth of questions that have been addressed around one critical HPS issue for LMICs, user fees; and the different papers examining the household level impacts of out of pocket payments together provide deeper and richer insights on these experiences than would come from one perspective alone.

Box 1. Drawing on Different Perspectives to Understand and Explain Experiences of User Fee Policy Change in Low- and Middle-Income Countries

Assessing household level impacts

Positivist perspectives:

  • What is the impact of out of pocket payments on household poverty levels across countries?
    1. ○ Cross-national statistical analysis [5] (health economics)
  • What is the impact of user fee removal on aggregate patient utilisation and across different patient socioeconomic groups within one country?
    1. ○ Before and after statistical analysis [24] (health economics)

Relativist perspectives:

  • How do pocket payments combine with other influences over health-seeking behaviour to impact on the dynamics of household poverty?
    1. ○ Mixed method study involving longitudinal household case studies [22] (development sociology, health economics)

Explaining policy implementation experiences

Critical realist and relativist perspectives:

  • What political forces led to user fee introduction/removal, and why was equity neglected as a policy goal?
    1. ○ Qualitative study [25] (social anthropology, policy analysis)
  • How does the process of implementing user fee removal influence health worker morale?
    1. ○ Multiple method study within overarching qualitative approach [26] (sociology, policy analysis)
  • How is the process of implementing user fees, in interaction with other policies, influenced by wider societal forces?
    1. ○ Ethnographic study [27] (anthropology)

Learning from Relativist Social Science Perspectives

Health policies and systems are fundamentally shaped by political decision-making, whilst the routines of health systems are brought alive through the relationships among the actors involved in managing, delivering, and accessing health care, and engaged in wider action to promote health, including researchers [11]. In essence, therefore, health policies and systems are constructed through human behaviour and interpretation, rather than existing independently of them. As relativist social science perspectives see all phenomena as at least partially constructed in this way, they have particular value in building the methodological foundations of HPSR. Three contributions are discussed here: generalising from rich contextual understanding; supporting policy learning; and approaches to ensuring research rigour.

Taking Account of Context in Drawing out Generalisations

Multiple contextual factors influence the working of health systems. Health worker motivation, for example, reflects a range of personal, organisational, and societal factors, including relationships with others, and itself influences many aspects of the provision of health care. Similarly, patients' decisions to use services, or adhere to treatment advice, are responses to many contextual factors: their own understandings of illness, and how best to treat it; advice received from friends and family; past experience of health providers; the availability of cash to cover costs; and the gender dynamics influencing household decision-making. There are also multiple interpretations of the same experience as different people bring different contexts to bear on its interpretation. Health workers, for example, respond differently to the same financial incentive, and patients vary in their response to treatment advice. The causal mechanisms underpinning the changes brought about by new health policies or health system interventions are, thus, complex.

As a result, investigation of HPS issues demands research that seeks to understand and explain experiences by reference to the many layers of their context, whilst acknowledging the often quite different interpretations of experience across people. Reducing relevant contextual factors to a set of simple quantifiable measures for statistical analysis is, simply, difficult. On the other hand, case study research, widely used in organisational and political science work, supports the “thick descriptions” of particular experiences situated within their context that allow understanding and explanations of the phenomena of focus by reference to that context [12]. For example, a study of Brazilian health system decentralisation, involving anthropological work in three case study areas, investigated the factors shaping the extent of local decision-making actually achieved, with consequences for quality of care improvement possibilities. A range of contextual factors were influential, including political relationships among layers of government, the potential of generating tax revenue at the local level, differences between rural and urban areas in the opportunities for community participation in decision-making, and existing patterns of political patronage; and these also combined with individual management styles and health worker commitment to the local area [13].

In studies with multiple cases, systematic and deliberate cross-case comparison supports, moreover, analytic generalisation (Box 2). The aim in such analysis is not to draw conclusions that can be statistically generalised to a wider study population, or that will hold across time and place. Instead, analytic generalisation entails the development of general conclusions that, although derived from a limited number of particular experiences, provide theoretical insights that can be put forward for consideration, and testing, in other, similar situations. This includes middle range theory, as outlined earlier, and theory that offers ideas about the causal mechanisms likely to underpin interventions that achieve their goals.

Box 2. An Example of Analytic Generalisation [28]

A study of the factors underpinning successful family planning programmes involved work in eight country cases. In each country a rich description of the evolution of programme development over time was developed, based on qualitative interviews with policy elites and documentary data analysis.

The countries were paired on the basis of similar socioeconomic development, but in each pair one country had a strong and one a weak, family planning programme. Comparison of experience within and across pairs, suggested that governments' commitment to family planning programmes was influenced by the process of their development and implementation.

More specifically, three factors were identified as likely to underpin successful family planning programmes: coalitions among elite groups with influence over health policy, that support effective programme development; spreading the risk associated with the sensitive issue of family planning among groups and over time; and having a clear and stable organisational structure in charge of implementation, as well as adequate funding. These conclusions were the general insights put forward for consideration and testing in other settings.

Active Support for Policy Learning

Health research has traditionally seen knowledge generation as essentially a process of adding to the existing stock of facts and predictions, with researchers acting largely as disinterested scientists feeding evidence into the decision-making process [14]. Learning from that knowledge then entails the simple transfer of knowledge from one setting to another [15]. Even current HPSR debates about the importance of getting research into policy and practice and knowledge translation sometimes see this process as quite linear [16].

However, for a relativist, researchers contribute to the process of learning as active participants, using both formal and tacit knowledge in active debate with policy makers [15]. Thus, some social scientists argue that in addressing problems that matter in their own communities, researchers should pay particular attention to the ways in which values and power shape those problems and responses to them [17], assisting policy actors to negotiate mutually acceptable solutions to problems, and ensuring that underrepresented groups are heard [18]. For others, building the possibility of such action into research design is an ethical requirement and key hallmark of good quality research [19].

Social science perspectives, therefore, challenge the HPSR community to think more deeply about how to support policy and system change through their research, including how to address the thorny issue of the boundary between researcher and advocate. For example, what sorts of participatory and action research with citizens, health managers, and health workers can support the reflective enquiry that generates positive change in current practices? And should and can we initiate processes that stimulate public debate about research findings—such as active media engagement, debates on public platforms, or engagement with civil society organisations?

Ensuring Research Rigour

For some traditions of health research, validity and reliability are the hallmarks of rigorous research, and are ensured through careful study design, appropriate tool development and data collection, and correct approaches to statistical analysis. In contrast, relativist (qualitative) social science research is premised on the understanding that there are multiple realities, reflecting actors' different understandings of common experiences (Figure 1). These understandings are either seen to have significant influence over the issues of focus or to be the focus of inquiry. Researchers from this tradition, moreover, aim not just to identify and report such understandings, but instead, through analysis and engagement, to produce their own interpretations of them, explaining why and how actors behave and think as they do. For relativist research, the “trustworthiness of researchers'” interpretations is the key hallmark of research rigour, implying that the interpretation is widely recognised to have value beyond the particular examples considered. Such trustworthiness is, in essence, negotiated between researchers and research users on the basis of transparent information on study design and the processes of data collection, analysis, and interpretation. Table 1 summarises the critical steps researchers must take to ensure that their analysis is both based on rich insight into the experience examined and has been subject to challenge, and to offer a transparent account of their research process to the user.

thumbnail

Table 1. Processes for ensuring rigour in case study and qualitative data collection and analysis [20],[29].

doi:10.1371/journal.pmed.1001079.t001

At a minimum, improving the quality of HPSR requires paying due attention to the particular approaches to research rigour relevant to the specific paradigm of knowledge underpinning any study. However, because of the complexity of the issues investigated, social science perspectives on rigour offer valuable insights for all empirical HPSR. As HPSR is often more investigation than observation, all stages of research must always be conducted with caution. Rigorous investigation involves the following [19][21]:

  • an active process of questioning and checking in inquiry—asking how and why things happened and not only what happened, checking answers to questions to identify further issues that need to be followed up to deepen understanding of the experience;
  • a constant process of conceptualising and reconceptualising—using ideas and theory to develop an initial understanding of the problem or situation of focus to guide data collection, but using the data collected to challenge those ideas and assumptions and when necessary, to revise your ideas in response to the evidence;
  • crafted, interpretative judgements—based on enough evidence, particularly about context, to justify the conclusions drawn, as well as deliberate consideration of contradictory evidence (negative case analysis) and review of initial interpretations by respondents (member checking);
  • researcher reflexivity—being explicit about how your own assumptions may influence your interpretation, and testing them in analysis.

Finally, although currently rarely conducted in HPSR, mixed-method research in which qualitative and quantitative analyses are undertaken sequentially, with one stage of work deliberately feeding into the next [22], offer important opportunities for the triangulation across methods and knowledge paradigms that can broaden and deepen investigation of health policy and systems issues [23].

Conclusions

The current interest in HPSR provides exciting opportunities for the field, but also brings the threat of “disciplinary capture” by the clinical, biomedical, and epidemiological disciplinary perspectives dominant in wider health research. Yet, social science perspectives are vital to HPSR. Health policies and systems are complex social and political phenomena, constructed by human action rather than naturally occurring. Advancing the science of HPSR, thus, demands we take steps to build understanding across disciplinary boundaries, for example, by ensuring that we can speak each other's languages around generalisability and knowledge generation; sharing experience of supporting policy learning; and clarifying expectations of each other's disciplinary culture. Valuing social science perspectives and building interdisciplinary understanding both represents the cutting edge of HPSR and demonstrates that the field is at a scientific cutting edge.

Author Contributions

Wrote the first draft of the manuscript: LG KH KS IA FS SB. Contributed to the writing of the manuscript: LG KH KS IA FS SB. ICMJE criteria for authorship read and met: LG KH KS IA FS SB. Agree with manuscript's results and conclusions: LG KH KS IA FS SB.

References

  1. 1. Sheikh K, Gilson L, Agyepong IA, Hanson K, Ssengooba F, Bennett S (2011) Building the field of Health Policy and Systems Research: framing the questions. PLoS Med 8: e1001073. doi:10.1371/journal.pmed.1001073.
  2. 2. Mills A (2011) Health policy and systems research: defining the terrain; identifying the methods. Health Policy Plan 2011: 1–7. E-pub ahead of print: 15 February 15 2011.
  3. 3. Reich SM, Reich JA (2006) Cultural competence in interdisciplinary collaborations: a method for respecting diversity in research partnerships. Am J Community Psychol 38(1–2): 51–62.
  4. 4. Bennett S, Agyepong IA, Sheikh K, Hanson K, Ssengooba F, Gilson L (2011) Building the field of Health Policy and Systems Research: an agenda for action. PLoS Med 8: e1001081. doi:10.1371/journal.pmed.1001081.
  5. 5. van Doorslaer E, O'Donnell O, Rannan-Eliya RP, Somanathan A, Adhikari S, et al. (2006) Effects of payments for health care on poverty estimates in 11 countries in Asia: an analysis of household survey data. Lancet 368(9544): 1357–1364.
  6. 6. Fernald LCH, Gertler PJ, Neufeld LM (2009) 10-year effect of Opportunidades, Mexico's conditional cash transfer programme, on child growth, cognition, language, and behaviour: a longitudinal follow-up study. Lancet 374: 1997–2005.
  7. 7. Riewpaiboon W, Chuengsatiansup K, Gilson L, Tangcharoensathien V (2005) Private obstetric practice in a public hospital: mythical trust in obstetric care. Soc Sci Med 61: 1408–1417.
  8. 8. Sheikh K, Porter J (2010) Discursive gaps in the implementation of public health policy guidelines in India: the case of HIV testing. Soc Sci Med 71(11): 2005–2013.
  9. 9. Shiffman J (2009) A social explanation for the rise and fall of global health issues. Bulletin of the World Health Organization. Bull World Health Organ 87(8): 608–613.
  10. 10. Marchal B, Dedzo M, Kegels G (2010) A realist evaluation of the management of a well- performing regional hospital in Ghana. BMC Health Services Research 10: 24. doi:10.1186/1472-6963-10-24.
  11. 11. Ssengooba F, Rahman SA, Hongoro C, Rutebemberwa E, Mustafa A, et al. (2007) Health sector reforms and human resources for health in Uganda and Bangladesh: mechanisms of effect. Human Resources for Health 5: 3. Available: http://www.human-resources-health.com/co​ntent/5/1/3. Accessed 21 July 2011.
  12. 12. Yin RK (2009) Case study research: design and methods. 4th edition. Thousand Oaks: Sage.
  13. 13. Atkinson S, Medeiros RLR, Oliveria PHL, de Almeida RD (2000) Going down to local: incorporating social organisation and political culture into assessments of decentralised health care. Soc Sci Med 51(4): 619–636.
  14. 14. Walt G (1994) How far does research influence policy? Eur J Public Health 4(4): 233–235.
  15. 15. Freeman R (2006) Learning in public policy. In: Moran M, Rein M, Goodin R, editors. Chapter 16. The Oxford handbook of public policy. Oxford: Oxford University Press.
  16. 16. Lavis JN, Lomas J, Hamid M, Sewankambo NK (2006) Assessing country level efforts to link research to action. Bull World Health Organ 84(8): 630–628.
  17. 17. Flyvberg B (2001) Making social science matter: why social inquiry fails and how it can succeed again. Cambridge: Cambridge University Press.
  18. 18. Yanow D (2000) Conducting interpretive policy analysis. Newbury Park (CA): Sage.
  19. 19. Henning E (2004) Finding your way in qualitative research. Pretoria: Van Schaik Publishers.
  20. 20. Pope C, Mays N (2009) Critical reflections on the rise of qualitative research. BMJ 339: b3425.
  21. 21. Thomas A, Chataway J, Wuyts M, editors. (1998) Finding our fast: investigative skills for policy and development. London, Thousand Oaks, New Dehli: Sage Publications.
  22. 22. Russell S, Gilson L (2006) Are health services protecting the livelihoods of the urban poor in Sri Lanka? Findings from two low-income areas of Colombo. Soc Sci Med 63(7): 1732–1744.
  23. 23. Sandelowski M (2000) Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res Nurs Health 23: 246–255.
  24. 24. Xu K, Evans DB, Kadama P, Nabyonga J, Ogwal PO, et al. (2006) Understanding the impact of eliminating user fees: utilization and catastrophic health expenditures in Uganda. Soc Sci Med 62(4): 866–876.
  25. 25. Ridde V (2007) “The problem of the worst-off is dealt with after all other issues”: the equity and health policy implementation gap in Burkina Faso. Soc Sci Med 66(6): 1368–1378.
  26. 26. Walker L, Gilson L (2004) We are bitter but we are satisfied: nurses as street level bureaucrats in South Africa. Soc Sci Med 59(6): 1251–1261.
  27. 27. Foley EE (2001) No money, no care: women and health sector reform in Senegal. Urban Anthropology 30(1): 1–50.
  28. 28. Lee K, Lush L, Walt G, Cleland J (1998) Family planning policies and programmes in eight low income countries: A comparative policy analysis. Soc Sci Med 47(7): 949–959.
  29. 29. Robson C (2002) Real world research: a resource for social scientists and practitioner-researchers. Second edition. Oxford: Blackwell Publishing.
  30. 30. Gilson L, Khumalo G, Erasmus E, Mbatsha S, McIntyre D (2004) Exploring the influence of workplace trust over health worker performance: preliminary national overview report South Africa. Johannesburg: Centre for Health Policy. Unpublished report.