Most maternal deaths, intrapartum-related stillbirths, and newborn deaths in low income countries are preventable but simple, effective methods for improving safety in institutional births have not been devised. Checklist-based interventions aid management of complex or neglected tasks and have been shown to reduce harm in healthcare. We hypothesized that implementation of the WHO Safe Childbirth Checklist program, a novel childbirth safety program for institutional births incorporating a 29-item checklist, would increase delivery of essential childbirth practices linked with improved maternal and perinatal health outcomes.
Methods and Findings
A pilot, pre-post-intervention study was conducted in a sub-district level birth center in Karnataka, India between July and December 2010. We prospectively observed health workers that attended to women and newborns during 499 consecutively enrolled birth events and compared these with observed practices during 795 consecutively enrolled birth events after the introduction of the WHO Safe Childbirth Checklist program. Twenty-nine essential practices that target the major causes of childbirth-related mortality, such as hand hygiene and uterotonic administration, were evaluated. The primary end point was the average rate of successful delivery of essential childbirth practices by health workers. Delivery of essential childbirth-related care practices at each birth event increased from an average of 10 of 29 practices at baseline (95%CI 9.4, 10.1) to an average of 25 of 29 practices afterwards (95%CI 24.6, 25.3; p<0.001). There was significant improvement in the delivery of 28 out of 29 individual practices. No adverse outcomes relating to the intervention occurred. Study limitations are the pre-post design, potential Hawthorne effect, and focus on processes of care versus health outcomes.
Introduction of the WHO Safe Childbirth Checklist program markedly improved delivery of essential safety practices by health workers. Future study will determine if this program can be implemented at scale and improve health outcomes.
Citation: Spector JM, Agrawal P, Kodkany B, Lipsitz S, Lashoher A, et al. (2012) Improving Quality of Care for Maternal and Newborn Health: Prospective Pilot Study of the WHO Safe Childbirth Checklist Program. PLoS ONE 7(5): e35151. doi:10.1371/journal.pone.0035151
Editor: Philippa Middleton, The University of Adelaide, Australia
Received: January 24, 2012; Accepted: March 8, 2012; Published: May 16, 2012
Copyright: © 2012 Spector et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by grants from World Health Organization (WHO) and the Children's Investment Fund Foundation (CIFF). WHO provided input on study design, data analyses, decision to publish, and preparation of the manuscript. WHO had no role in data collection. CIFF had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: AAG receives royalties for books and essays focused on patient safety and medicine. CL receives royalties for a book focused on healthcare management. All other authors have declared that no competing interests exist. This does not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials.
Reducing childbirth-associated mortality is a top global health priority but simple, effective methods to achieve it are severely lacking. Most of the 350,000 maternal deaths, 1·2 million intrapartum-related stillbirths, and 3·1 million neonatal deaths that occur each year could be avoided through the delivery of timely interventions proven to be effective and affordable , , , , . Shifting place of delivery from home to hospital is a key strategy for improving childbirth outcomes and has led to unprecedented increases in institutional births in several countries , , . But even as institutional birth rates rise, morbidity and mortality rates have been slow to fall , , . Poor quality care in institutional births is recognized to be a major contributing factor to childbirth-related harm , , . Improving facility-based care is a critical necessity yet no widely applicable, effective method currently exists.
In recent years checklist-based interventions have been adopted with increasing frequency in health to aid management of complex or neglected tasks that risk serious human harm. Integration of checklist programs into clinical practice has been shown to reduce deaths and complications in intensive care medicine and surgery , , , . Several features of childbirth make a checklist-based strategy promising: the major causes of maternal and perinatal mortality are well described; most deaths occur within a narrow time window (twenty-four hours after birth); international guidelines for best practices exist but are not followed; and proven interventions are relatively inexpensive and easy to perform, but can be difficult to remember and execute in proper sequence , , .
In 2008, the World Health Organization (WHO) established a checklist-based childbirth safety program with the goal of determining whether a simple, low-cost, scalable intervention with potential for broader testing could be devised. A 29-item bedside WHO Safe Childbirth Checklist was developed according to previously established methodology and tested for usability in ten countries in Africa and Asia . An implementation program was designed to maximize the likelihood of successful checklist adoption into clinical practice. We hypothesized that this program would increase the rate of delivery of essential childbirth practices linked with improved maternal, fetal, and neonatal health outcomes in a low income setting.
We conducted a prospective, pre-post-intervention study observing childbirth practices of health workers at a sub-district level hospital in Karnataka, India. Our plans were to observe health workers attending to a minimum of 300 birth events before the intervention; introduce the checklist program; and then monitor health workers attending to a minimum of 300 birth events after the intervention. The total study was anticipated to last 6 months. The pilot hospital was selected on the basis of sufficient birth volume (minimum 250 births monthly), general availability of supplies, motivated leadership, and absence of other ongoing interventions. Basic emergency obstetric care and caesarean sections are offered at the facility . Nurses provide care during most births. Other staff includes two obstetricians, one surgeon who performs caesarean sections, and an anesthetist. There are no pediatricians on staff. A co-investigator (BK) led the project locally and the hospital administration endorsed the intervention. Five data collectors were trained by the investigators to observe and document health worker practices. Data collectors were student nurses previously unknown to hospital staff with no clinical responsibilities. Inclusion criteria were health workers at the study site who cared for women and newborns from the time of admission for childbirth to discharge. Health workers providing care to mothers not involved in childbirth, including those being managed for abortions, miscarriages, and antenatal problems were excluded. Written informed consent was obtained from health workers and patients. The study was approved by ethics committees at JNMC Medical College in Karnataka, WHO, and the Harvard School of Public Health. The Indian Council of Medical Research also approved the study.
The intervention was a four-step checklist-based childbirth safety program designed and implemented using methods adapted from previous programs; in particular, CUSP and TeamSTEPPS , . It involved (1) Engagement of local administrative and clinical leaders and identification of facility-based implementation leads; (2) Education about childbirth safety principles, deficiencies in current practice, and how to use the WHO Safe Childbirth Checklist during a one-day learning session; (3) Execution beginning with one week of simulation and supervised practice; and (4) Evaluation and ongoing monitoring .
The hospital-based implementation leads (an administrator, a senior physician, and a senior labor nurse) were selected by the facility and trained by the investigators. They introduced the checklist program to staff during the one-day learning session and monitored its ongoing use. Learning was supported by written materials, lectures, an instructional video, and hands-on simulation. Following introduction of the program, the implementation leads supervised use of the checklist and offered strategies for improvement in performance and learning when their full-time clinical schedules allowed. An outside physician “coach” visited the facility fortnightly to provide additional support to help staff improve adherence to the checklist program.
The WHO Safe Childbirth Checklist consists of succinct reminders of essential steps for safe childbirth care (see Table S1). Items on the checklist address the major causes of maternal deaths (i.e., hemorrhage, infection, obstructed labor, and hypertensive disorders), intrapartum-related stillbirths (i.e., inadequate intrapartum care), and neonatal deaths (i.e., intrapartum-related events, infection, and complications of prematurity) in lower income countries , , , . Checklist items are organized for use at four critical junctures in care during birth: at the time the woman is admitted, when the woman begins to push or before cesarean, within one hour after birth, and before discharge. Modifications to the checklist are encouraged to align content with local practice. Adjustments made by the pilot site's review committee were the following: change from “does mother need referral” to “does mother need review by obstetrician,” minor change to maternal antibiotic administration criteria (specifying labor >24 hours in a primigravida or labor >12 hours in multipara as indications), minor change to maternal magnesium sulfate administration criteria (specifying diastolic blood pressure threshold at 100 mmHg instead of 110 mmHg), inclusion of ASHA (Accredited Social Health Activist) workers as satisfactory birth companions, requirement that nurses introduce themselves by name to laboring women, removal of the newborn special care and monitoring item (all ill newborns were referred to other facilities since there was no pediatrician on staff), and removal of the follow-up after discharge item (the existing follow-up process was felt by local staff to be sufficient). No additional equipment, supplies, or medications were provided. Checklist use was not mandatory; individual health workers could decide for themselves whether or not to use the checklist during any given patient encounter. Completed checklists were attached to the mothers' charts.
Data were collected through observation of health workers' practices and review of birth registers. Observation data were recorded on standardized data sheets by data collectors who directly observed health workers. Observation took place 24 hours daily for a minimum of six days weekly; unobserved days were selected at random. Practical limitations precluded continuous observation of each woman from the time of admission until discharge. Observation therefore took place at three specific periods: on admission, continuously from the time of pushing until one hour after delivery, and before discharge. Checklist use was observed during the post-intervention period. Data collectors did not interact with patients or health workers during their observations. For ethical reasons, they were instructed to notify health workers if they observed a potentially harmful condition or practice. Data quality was assured through periodic assessment of data collector skills (confirming they achieved 100% concordance on a sample of three observations), parallel observations by the local study coordinator once weekly, and on-site review of all observation forms within 72 hours. The data management system had range, plausibility, and cross-validation checks confirming all data were logical. Double data entry was performed for a sample of the observation forms.
The primary outcome was the average rate of successful delivery of essential childbirth practices by health workers at each birth event. A birth event was defined as the period from admission for childbirth to discharge. Twenty-nine practices were evaluated. Successful delivery of individual practices was defined by completion of a predetermined set of process indicators (see Table S2). For several practices this required proper assessment of the mother or baby, recognition of abnormal signs or symptoms, and execution of appropriate action. Essential childbirth practices relating to maternal care were referral when indicated, partograph use, periodic assessment of infection risk and antibiotic administration when indicated, periodic assessment of hypertensive-disease risk and magnesium sulfate administration when indicated, assessment of HIV risk and anti-retroviral administration when indicated, presence of a birth companion, good hand hygiene, periodic danger sign counseling, presence of an assistant for birth, oxytocin administration within 1 minute after birth, periodic blood loss estimation and bleeding risk, and family planning discussion. Essential childbirth practices relating to newborn care were use of a sterile blade to cut the umbilical cord, proper thermal and resuscitation care, referral when indicated, assessment of HIV risk and anti-retroviral administration when indicated, breastfeeding within one hour, periodic assessment of infection risk and antibiotic administration when indicated, assessment of adequate feeding before discharge, and periodic danger sign counseling.
Observed rates of in-facility maternal deaths, newborn deaths, and stillbirths were analyzed as secondary outcomes. The frequency of medication administration before and after the intervention was also measured.
We aimed to collect data on a minimum of 300 consecutively enrolled birth events during each phase of the study. The sample size was calculated to detect a 20% absolute increase in the average rate of essential childbirth practices successfully delivered by health workers after introduction of the checklist program, with a statistical power of 80% and an alpha value of 0.05, using a generalized estimating equations (GEE) test accounting for clustering by provider , .
Each of the 29 individual childbirth practices were quantified in terms of the proportion successfully delivered, and GEE methods were again used to adjust confidence intervals to account for clustering by provider. Rao-Scott Chi-Square tests were used to test whether possible categorical confounders (categorical patient characteristics) had different distributions before and after the intervention, accounting for clustering by provider. If covariates were found to be imbalanced over the two phases, regression analyses were conducted to adjust for possible confounding due to patient characteristics.
Using the Bonferroni approach to adjust the overall 5% Type I Error rate for the 30 before versus after comparisons (each of the 29 individual childbirth practices plus the average rate of practices delivered), a P-value less than 0.05/30 = 0.0017 would be declared significant.
Our secondary outcome was the change in observed rates of in-facility maternal deaths, newborn deaths, and stillbirths, although the study was not powered to detect a difference in mortality. Secondary outcomes were exploratory and the Type I error rate was not adjusted.
Analyses were performed with SAS version 9·2 (SAS Institute Inc, Cary, North Carolina).
We observed health workers attending to 499 birth events during the baseline period (July–September, 2010) and 795 birth events after introduction of the checklist program (September–December, 2010). All hospital staff involved in childbirth were invited to participate and agreed to do so, and there was no staff turnover during the study. The pre- and post-intervention periods did not overlap and data was not collected during the brief period when the program was introduced. Table 1 lists the number of observations made at each period in the flow of care. Demographic characteristics of women and newborns are shown in Table 2; there were no differences in the two phases of the study. The checklist was observed to be used by health workers at least 95% of the time at each of the four checklist pause points in the post-intervention period.
Table 1. Number of childbirth events observed at each period before and after intervention.doi:10.1371/journal.pone.0035151.t001
Table 2. Demographic characteristics of women and newborns before and after intervention.doi:10.1371/journal.pone.0035151.t002
The rate of successful delivery of essential practices at each birth event increased from an average of 10 of 29 practices at baseline (95%CI 9.4, 10.1) to an average of 25 of 29 practices afterwards (95%CI 24.6, 25.3; p<0.001)(Figure 1). The Bonferroni correction did not affect the outcomes that were declared significant since all P-values less than 0.05 were also less than 0.0017. Figure 2 shows the rates of successful completion of individual practices before and after introduction of the checklist program. There was significant improvement in the delivery of every practice except maternal referral.
Figure 1. Average rate of successful delivery of essential childbirth practices before and after intervention (p<0.001).doi:10.1371/journal.pone.0035151.g001
Figure 2. Changes in rates of delivery of specific childbirth practices before and after intervention; (2a) On admission; (2b) From pushing until delivery; (2c) Soon after birth (within one hour); (2d); Before discharge (*P value = 0.052; all others p≤0.001).doi:10.1371/journal.pone.0035151.g002
Figure 3 shows the frequency of initiation of medication therapy at each period of care. The use of medications increased at some periods and decreased at others. There were no differences in observed maternal and neonatal deaths in the two study periods, though the study was not powered to detect significant differences in mortality (Table 3). The stillbirth rate showed a declining trend after introduction of the checklist program.
Figure 3. Frequency of initiation of medication therapy before and after intervention (*p<0.05).doi:10.1371/journal.pone.0035151.g003
In this pilot study, a novel checklist-based childbirth safety program led to a marked increase in delivery of essential childbirth practices linked with improved maternal, fetal, and newborn outcomes. Overall, there was an average 150% increase in adherence to accepted clinical practices at any given birth event, and 28 of 29 individual practices were delivered with significantly greater frequency. Strategies for achieving quality improvements in institutional births in low income settings are lacking. These results suggest the potential for this approach to improve maternal and perinatal care.
While these results were found in just a single hospital, research in this area has not previously demonstrated interventions that are comprehensive (to capture sufficient behaviors to have a chance of saving lives) and simple (to be scalable and sustainable). Moreover, facilitating behavior change to increase adherence to evidence-based healthcare practices is known to be challenging , . In this initial work we aimed to prove that such methods could be devised and produce measurable change.
The ways by which a checklist-based approach improves care during childbirth warrants exploration to better understand how this intervention produced such promising results. We believe the intervention had three primary mechanisms of effect: (1) as a checklist instrument that reinforced for health workers a core set of essential practices that must be completed at each and every birth; (2) as a reminder to complete these practices at the most crucial period – at the bedside at the moment of care; and (3) as a tool that highlighted gaps in the existing system of care at the facility which enabled local staff to take steps to effectively strengthen their own health system to ensure adherence to checklist practices. No additional investments in equipment or supplies were made, and no incentives were given. The local team seemed inspired by the checklist program and developed a personal interest in helping it to succeed.
There were changes in behavioral patterns that were individual in nature, for example improvement in health workers washing their hands with soap and water and wearing clean gloves (by comparison, before the intervention soap was not used routinely). Other improvements resulted from system changes. For instance, after introduction of the program it became apparent that no structure was in place to adequately monitor women and newborns immediately after delivery, which brought to light the difficulty in reliably completing crucial assessments and practices at that time. In response, the local staff took the initiative to convert an underutilized room adjacent to the labor ward into a postpartum bay where women and newborns were observed for at least one hour after delivery. The checklist program also helped to identify similar deficiencies in the discharge process, which was subsequently systematized by the local staff. The program reinforced the importance of ensuring that medicines and supplies were readily available to health workers on the labor ward and not kept remotely in the hospital store. Finally, the intervention appeared to improve communication and teamwork. Nursing staff used the checklist as a framework for communicating patient information at shift changes as well as conducting daily safety rounds to discuss complicated cases as a team.
Medication use, which increased in some periods and decreased in others, was a partial contributor to improved practices. Improved assessment of mothers and newborns was often the most significant factor in health workers' successful adherence to essential practices. We suspect that increased medication usage reflected improved awareness of appropriate indications for administration (for instance, increased antibiotic administration to at-risk newborns in the postnatal period) and that decreased medication administration reflected a decline in overuse practices (for instance, a reduction in the traditional practice of administering antibiotics after birth for all episiotomies).
We recognize that a simple paper checklist alone is unlikely to result in lasting behavior change . In this study the WHO Safe Childbirth Checklist was the central component of an implementation program based on a well-described change model carried out by hospital administration and clinical leaders . The program involves engaging and empowering the local team; providing education on best practices and existing deficiencies; discussing potential barriers and introducing the checklist through focused training; and establishing a mechanism for ongoing monitoring and evaluation . This is a comprehensive behavior change strategy facilitated by a checklist program. We found that this approach was associated with rapid uptake by the local team and low implementation costs.
This study has several limitations. A risk of the pre-post-intervention design is confounding by secular trends. The study period was, however, limited to six months and no difference was observed in the characteristics of the women and newborns in the two phases of the study. No other interventions took place during the study period and there were no changes in hospital staffing. For these reasons, secular trends alone were unlikely to be responsible for the observed differences.
A potential Hawthorne effect, in which subjects' behavior is influenced by an awareness of being observed, is another methodological limitation given that independent observers conducted the evaluation of health workers' performance . This methodology was selected for its obvious advantages over self-reporting and for practicality (for instance, video cameras in this setting would have been infeasible). We worked to minimize Hawthorne effect by employing the same data collectors before and after the intervention, by having the health workers observed in the same way for both periods of observation, and by structuring nearly continuous periods of observation from the start of the study so that health workers had the opportunity to become accustomed to the presence of observers.
Additional concerns relate to the study's generalizability and sustainability. The pilot facility is representative of first-level referral centers in India albeit with a motivated local team and general accessibility to childbirth equipment and supplies. These factors undoubtedly contributed to implementation success. The potential efficacy of this approach in other settings is unclear and merits further study. Regarding sustainability, there is evidence from other disciplines to suggest that checklist programs remain to be associated with sustained health improvements and positive attitudes toward the programs up to at least 18 months after the initial introduction and evaluation periods , , . A follow-up to this pilot study is now being organized to assess checklist use and adherence to essential practices more than 12 months after the initial investigation.
Lastly, this pilot study focused on processes of care as indicators of quality. Though the observed stillbirth rates showed a declining trend, the sample size in this pilot investigation was insufficient to detect significant differences in mortality. A multi-center study in north India is currently underway to measure the impact of the program on a composite measure of severe maternal, fetal, and newborn harm. Enrollment for this prospective randomized trial is anticipated to begin in late 2012.
Most maternal and newborn deaths, and many stillbirths, are avoidable. Assuring the delivery of key evidence-based interventions during childbirth is critical to optimizing care for women and newborns and helping priority countries to achieve progress toward Millennium Development Goals Four and Five. In this study, implementation of a novel checklist-based childbirth safety program led to improved quality of care delivered by health workers attending to institutional deliveries. Future study is required to determine whether a checklist-based approach to promoting safety in childbirth also reduces harm and saves lives.
Elements of the WHO Safe Childbirth Checklist.
Definitions of childbirth practices used in the WHO Safe Childbirth Checklist program pilot study.
We are indebted to the individuals who contributed to the development the WHO Safe Childbirth Checklist program including: Sabaratnam Arulkumaran, Chris Barnes, Angela Bader, Dr Benachinamaradi, William Berry, Shereen Bhutta, Zulfiqar Bhutta, Ann Blanc, Dan Boorman, Waldemar Carlo, Susan Crowther, Gary Darmstadt, Louise Day, Ambassador Mark Dybul, Lizzie Edmondson, Barbara Farlow, Zhao Gengli, Kathleen Hill, Justus Hofmeyr, Narayan Honnungar, Julia Hussein, AK Jana, Cate Kamau, Grace Kodindo, Barbara Kwast, Rashad Massoud, Claudia Morrissey, Elizabeth Morse, Susan Niermeyer, Padmanaban Packirisamy, Vinod Paul, Robert Pattinson, Amit Revankar, Harshad Sanghvi, Susan Sheridan, Youssef Tawfik, Phommady Vesaphong, Wilson Were, Jean-José Wolomby, Blair Wylie, and Juliana Yartey. We also deeply thank the individuals that conducted early study of the checklist program for practical usability in their own hospitals, including Shereen Bhutta, Zulfiqar Bhutta, Jane Machira, and Leonard Okoko.
Conceived and designed the experiments: JMS AAG. Performed the experiments: JMS PA BK AL GD AAG. Analyzed the data: PA SL JMS AAG. Wrote the paper: JMS PA AAG BK SL AL GD RB M. Merialdi M. Mathai CL.
- 1. Hogan MC, Foreman KJ, Naghavi M, Ahn SY, Wang M, et al. (2010) Maternal mortality for 181 countries, 1980–2008: a systematic analysis of progress towards Millennium Development Goal 5. Lancet 375: 1609–1623.
- 2. Rajaratnam JK, Marcus JR, Flaxman AD, Wang H, Levin-Rector A, et al. (2010) Neonatal, postneonatal, childhood, and under-5 mortality for 187 countries, 1970–2010: a systematic analysis of progress towards Millennium Development Goal 4. Lancet 375: 1988–2008.
- 3. Cousens S, Blencowe H, Stanton C, Chou D, Ahmed S, et al. (2011) National, regional, and worldwide estimates of stillbirth rates in 2009 with trends since 1995: a systematic analysis. Lancet 377: 1319–1330.
- 4. WHO , UNICEF , UNFPA , WorldBank (2010) Trends in maternal mortality: 1990–2008. Geneva.
- 5. Lawn JE, Lee AC, Kinney M, Sibley L, Carlo WA, et al. (2009) Two million intrapartum-related stillbirths and neonatal deaths: where, why, and what can be done? Int J Gynaecol Obstet 107 Suppl 1: S5–18, S19:
- 6. Bullough C, Meda N, Makowiecka K, Ronsmans C, Achadi EL, et al. (2005) Current strategies for the reduction of maternal mortality. BJOG : an international journal of obstetrics and gynaecology 112: 1180–1188.
- 7. Lim SS, Dandona L, Hoisington JA, James SL, Hogan MC, et al. (2010) India's Janani Suraksha Yojana, a conditional cash transfer programme to increase births in health facilities: an impact evaluation. Lancet 375: 2009–2023.
- 8. Rana TG, Chataut BD, Shakya G, Nanda G, Pratt A, et al. (2007) Strengthening emergency obstetric care in Nepal: The Women's Right to Life and Health Project (WRLHP). International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics 98: 271–277.
- 9. Powell-Jackson T, Morrison J, Tiwari S, Neupane BD, Costello AM (2009) The experiences of districts in implementing a national incentive programme to promote safe delivery in Nepal. BMC health services research 9: 97.
- 10. Ekirapa-Kiracho E, Waiswa P, Rahman MH, Makumbi F, Kiwanuka N, et al. (2011) Increasing access to institutional deliveries using demand and supply side incentives: early results from a quasi-experimental study. BMC international health and human rights 11: Suppl 1S11.
- 11. Koblinsky M, Matthews Z, Hussein J, Mavalankar D, Mridha MK, et al. (2006) Going to scale with professional skilled care. Lancet 368: 1377–1386.
- 12. Friberg IK, Kinney MV, Lawn JE, Kerber KJ, Odubanjo MO, et al. (2010) Sub-Saharan Africa's mothers, newborns, and children: how many lives could be saved with targeted health interventions? PLoS medicine 7: e1000295.
- 13. van den Broek NR, Graham WJ (2009) Quality of care for maternal and newborn health: the neglected agenda. BJOG : an international journal of obstetrics and gynaecology 116: Suppl 118–21.
- 14. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, et al. (2009) A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 360: 491–499.
- 15. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, et al. (2006) An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 355: 2725–2732.
- 16. Neily J, Mills PD, Young-Xu Y, Carney BT, West P, et al. (2010) Association between implementation of a medical team training program and surgical mortality. Jama 304: 1693–1700.
- 17. de Vries EN, Prins HA, Crolla RM, den Outer AJ, van Andel G, et al. (2010) Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med 363: 1928–1937.
- 18. Ronsmans C, Graham WJ (2006) Maternal mortality: who, when, where, and why. Lancet 368: 1189–1200.
- 19. Lawn JE CS, Zupan J (2005) 4 million neonatal deaths: When? Where? Why? Lancet 365: 891–900.
- 20. Harvey SA, Blandon YC, McCaw-Binns A, Sandino I, Urbina L, et al. (2007) Are skilled birth attendants really skilled? A measurement method, some disturbing results and a potential way forward. Bulletin of the World Health Organization 85: 783–790.
- 21. Weiser TG, Haynes AB, Lashoher A, Dziekan G, Boorman DJ, et al. (2010) Perspectives in quality: designing the WHO Surgical Safety Checklist. Int J Qual Health Care 22: 365–370.
- 22. Paxton A, Bailey P, Lobis S, Fry D (2006) Global patterns in availability of emergency obstetric care. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics 93: 300–307.
- 23. Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB, et al. (2006) Creating high reliability in health care organizations. Health Serv Res 41: 1599–1617.
- 24. King HB, Battles J, Baker DP, Alonso A, Salas E, et al. (2008) TeamSTEPPS: Team Strategies and Tools to Enhance Performance and Patient Safety. In: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. Advances in Patient Safety: New Directions and Alternative Approaches (Vol 3: Performance and Tools). Rockville (MD).
- 25. Pronovost PJ, Berenholtz SM, Needham DM (2008) Translating evidence into practice: a model for large scale knowledge translation. BMJ 337: a1714.
- 26. Lawn JE, Blencowe H, Pattinson R, Cousens S, Kumar R, et al. (2011) Stillbirths: Where? When? Why? How to make the data count? Lancet 377: 1448–1463.
- 27. Black RE, Cousens S, Johnson HL, Lawn JE, Rudan I, et al. (2010) Global, regional, and national causes of child mortality in 2008: a systematic analysis. Lancet 375: 1969–1987.
- 28. Lee AC, Cousens S, Darmstadt GL, Blencowe H, Pattinson R, et al. (2011) Care during labor and birth for the prevention of intrapartum-related neonatal deaths: a systematic review and Delphi estimation of mortality effect. BMC public health 11: Suppl 3S10.
- 29. Zeger SL, Liang KY (1986) Longitudinal data analysis for discrete and continuous outcomes. Biometrics 42: 121–130.
- 30. Lipsitz SR, Molenberghs G, Fitzmaurice GM, Ibrahim J (2000) GEE with Gaussian estimation of the correlations when data are incomplete. Biometrics 56: 528–536.
- 31. Althabe F, Buekens P, Bergel E, Belizan JM, Campbell MK, et al. (2008) A behavioral intervention to improve obstetrical care. The New England journal of medicine 358: 1929–1940.
- 32. Grol R, Grimshaw J (2003) From best evidence to best practice: effective implementation of change in patients' care. Lancet 362: 1225–1230.
- 33. Bosk CL, Dixon-Woods M, Goeschel CA, Pronovost PJ (2009) Reality check for checklists. Lancet 374: 444–445.
- 34. Pronovost PJ, Weast B, Rosenstein B, Sexton JB, Holzmueller CG, et al. (2005) Implementing and validating a comprehensive unit-based safety programme. Journal of Patient Safety 1: 33–50.
- 35. Mayo E (1933) The human problems of an industrial civilization. New York: Macmillan.
- 36. Pronovost PJ, Goeschel CA, Colantuoni E, Watson S, Lubomski LH, et al. (2010) Sustaining reductions in catheter related bloodstream infections in Michigan intensive care units: observational study. BMJ 340: c309.
- 37. Nilsson L, Lindberget O, Gupta A, Vegfors M (2010) Implementing a pre-operative checklist to increase patient safety: a 1-year follow-up of personnel attitudes. Acta anaesthesiologica Scandinavica 54: 176–182.
- 38. Young-Xu Y, Neily J, Mills PD, Carney BT, West P, et al. (2011) Association between implementation of a medical team training program and surgical morbidity. Archives of surgery 146: 1368–1373.