The rapid expansion of voluntary medical male circumcision (VMMC) has raised concerns whether health systems can deliver and sustain VMMC according to minimum quality criteria.
Methods and Findings
A comparative process evaluation was used to examine data from SYMMACS, the Systematic Monitoring of the Voluntary Medical Male Circumcision Scale-Up, among health facilities providing VMMC across two years of program scale-up. Site-level assessments examined the availability of guidelines, supplies and equipment, infection control, and continuity of care services. Direct observation of VMMC surgeries were used to assess care quality. Two sample tests of proportions and t-tests were used to examine differences in the percent of facilities meeting requisite preparedness standards and the mean number of directly-observed surgical tasks performed correctly. Results showed that safe, high quality VMMC can be implemented and sustained at-scale, although substantial variability was observed over time. In some settings, facility preparedness and VMMC service quality improved as the number of VMMC facilities increased. Yet, lapses in high performance and expansion of considerably deficient services were also observed. Surgical tasks had the highest quality scores, with lower performance levels in infection control, pre-operative examinations, and post-operative patient monitoring and counseling. The range of scale-up models used across countries additionally underscored the complexity of delivering high quality VMMC.
Greater efforts are needed to integrate VMMC scale-up and quality improvement processes in sub-Saharan African settings. Monitoring of service quality, not just adverse events reporting, will be essential in realizing the full health impact of VMMC for HIV prevention.
Citation: Jennings L, Bertrand J, Rech D, Harvey SA, Hatzold K, et al. (2014) Quality of Voluntary Medical Male Circumcision Services during Scale-Up: A Comparative Process Evaluation in Kenya, South Africa, Tanzania and Zimbabwe. PLoS ONE 9(5): e79524. doi:10.1371/journal.pone.0079524
Editor: Nathan Ford, World Health Organization, Switzerland
Received: July 28, 2013; Accepted: October 2, 2013; Published: May 6, 2014
This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Funding: The article was supported by the U.S. Agency for International Development (USAID) | Project SEARCH, Task Order No.2, funded by the USAID (www.usaid.gov) under Contract No. GHH-I-00-07-00032-00, beginning September 30, 2008, and supported by the President's Emergency Plan for AIDS Relief. The Research to Prevention (R2P) Project is led by the Johns Hopkins Center for Global Health and managed by the Johns Hopkins Bloomberg School of Public Health Center for Communication Programs (CCP). USAID staff played a role in study design and data analysis, decision to publish, and preparation of the manuscript. USAID staff did not play a role in data collection. The views expressed in this manuscript do not represent USAID's or the United States government's opinion.
Competing interests: The authors have declared that no competing interest exist.
Given the scientific evidence of the benefits of male circumcision in reducing HIV incidence rates among men –, several sub-Saharan African countries have begun to scale-up voluntary medical male circumcision (VMMC) services for adult and adolescent males –. High coverage of safe VMMC services could prevent up to 3.4 million new HIV infections . However, the rapid expansion of VMMC programs has raised questions regarding the extent to which health systems can deliver VMMC according to minimum quality standards which are sustained at-scale , . Ensuring correct performance of VMMC procedures during scale-up is essential in realizing the potential health impact of averted HIV infections and in reducing health systems costs for HIV-related care , .
In 2009, the World Health Organization (WHO) developed a quality assessment toolkit to assess a range of quality standards relating to facilities' VMMC supply inventory, provider competence, surgical technique, infection prevention, and continuity of care . Preliminary findings suggest that VMMC surgical complications have resulted from inadequate provider competence and training, poor clinical infrastructure, and limited patient information and follow-up , . Yet, there is a dearth of published data on other aspects of service quality among VMMC programs in resource-poor settings , , . Most quality assessments have focused on occurrence of adverse events , –, device malfunction rates , procedure duration , or patient acceptability and resumption of sexual activity . Less attention has been given to the availability of minimum service components and provider performance of specific surgical tasks in contexts where countries are rapidly expanding the number of VMMC sites , , . To our knowledge, this study is the first published to-date to address this gap by examining quality of VMMC services among four African countries over two years of program scale-up.
A comparative process evaluation was used to examine VMMC service provision over time. Data were drawn from SYMMACS, the Systematic Monitoring of the Voluntary Medical Male Circumcision Scale-Up, which assessed the evolution of VMMC scale-up in Kenya, South Africa, Tanzania, and Zimbabwe. SYMMACS conducted a two-stage multi-country evaluation with two cross-sectional data collection periods (one in 2011 and one in 2012) among health facilities providing VMMC and selected VMMC surgeries. A detailed description of the SYMMACS evaluation and sampling methodology is provided in this supplement .
The scale-up model used to expand the number of VMMC sites varied by country. Kenya and Tanzania adopted a focused implementation strategy using outreach sites and fixed clinical facilities supported by multiple implementing agencies within regions with the highest HIV prevalence and lowest male circumcision rates. Kenya additionally provided services via mobile sites to deliver VMMC in underserved areas. South Africa adopted a national approach using outreach and fixed sites with multiple partners, while Zimbabwe's national program was implemented by public sector service providers in outreach and fixed sites supported by a single implementing partner. Kenya was the only country with a fully-scaled VMMC program in the evaluation's first and second years, while Tanzania, South Africa, and Zimbabwe substantially increased the number of VMMC sites from two- to five-fold over the evaluation period.
The comparative process evaluation measured two outcomes: VMMC facility preparedness and quality of VMMC surgeries. VMMC facility preparedness was assessed by direct observation using a pre-tested observation checklist based on the WHO VMMC quality assessment tool . The checklist included items grouped into five preparedness categories: “guidelines and protocols” comprised 3 standard operating protocols; “equipment and supplies” included 9 items relating to anesthesia, antibiotics, condoms, and other supplies; “basic life support” (BLS) consisted of 4 items: cardiopulmonary resuscitation bag and mask, oxygen, intravenous lines, and anaphylaxis medications; “hygienic and safe infrastructure” consisted of 4 items measuring adequacy of lighting, ventilation, cleanliness, and adverse event monitoring; and “patient services” included 5 items: circumcision, group education, individual counseling, HIV counseling and testing, and referral services. Quality of VMMC surgeries was evaluated by direct observation of randomly selected surgeries using a checklist of tasks across three surgical periods: pre-operative (1 item), operative (21 items), and post-operative (7 items). Example tasks included pre-operative examination, use of sterile processes, correct surgical techniques, and provision of post-operative monitoring and counseling. Operative period tasks were further separated into two categories: ‘safe operative procedures’ (8 items) relating to surgical safety and ‘operative techniques’ (13 items) relating to step-by-step circumcision procedures. Trained observers scored items as “unsatisfactory” (scored = 0), “partially satisfactory” (scored = 1), and “satisfactory” (scored = 2).
Data were analyzed using STATA (Version 12). Two sample tests of proportions and t-tests were used to examine differences in facility preparedness and VMMC service quality by country between 2011 and 2012. Composite measures of facility preparedness and service quality were calculated based on the percent of facilities with all requisite items per category and the mean number and mean percent of items scored as “satisfactory” per category. Item-level differences between time points were also assessed using Pearson's chi-squared tests. A significance level of p<.05 was used in all analyses.
Ethical approval was obtained from the Tulane University Institutional Review Board (United States), the Kenya Medical Research Institute, the University of Witwatersrand's Human Research Ethics Committee (South Africa), the Tanzanian National Institute for Medical Research, and the Medical Research Council of Zimbabwe.
VMMC Facility Preparedness
Table 1 presents changes in the percent of facilities having all preparedness items per category. During the initial year, facilities in all four countries had an average of 1.3 to 2.0 guidelines on-site at the time of assessment. Across countries, less than half of facilities (Kenya: 34.5%, South Africa: 33.3%, Tanzania: 7.7%, Zimbabwe: 7.1%) had all three guidelines. In the second year, the percent of facilities having all three guidelines significantly improved in Zimbabwe (+34.6, 95% CI: 10.6, 58.4). However, no significant changes occurred in the remaining countries: Kenya (+19.1, 95% CI: −6.2, 44.4), Tanzania (+19.9, 95% CI: −1.9, 41.7), South Africa (−20.8, 95% CI: −46.8, 5.1).
Table 1. Summary of Availability of VMMC Protocols, Equipment, Supplies, and Services from Initial Year (2011) to Expansion Year (2012).doi:10.1371/journal.pone.0079524.t001
Health facilities in all four countries reported having most of the 9 listed equipment and supplies during the initial year (Kenya: 7.8, South Africa 8.3, Tanzania 8.2, Zimbabwe 7.9). With the exception of South Africa, roughly a quarter of facilities had all 9 items (Kenya 25%, South Africa 50%, Tanzania 23%, Zimbabwe 29%). As compared to the expansion year, availability of supplies and equipment significantly declined in South Africa from 50.0% of facilities having all 9 items to 2.6% of facilities (−47.4, 95% CI: −74.1, −20.8). No statistically significant changes were observed in the other countries, although the general trend was a decline in supply availability: Kenya (−14.3, 95% CI: −34.0, 5.4), Tanzania (−16.2, 95% CI: −40.9, 8.9), and Zimbabwe (+17.2, 95% CI: −13.7, 48.2). Sites in Kenya and Tanzania scored poorly on available BLS equipment, ranging from an average of 0.8 to 1.5 of the 4 recommended items. These levels increased slightly, but remained low during the expansion year, with 3.4% and 17.2% of facilities having all 4 BLS items. Although more facilities in South Africa (27.5%) had all 4 BLS items, this represented a significant decline from 64.3% of facilities in its initial year (−36.8, 95% CI:−65.4, −8.1). In Zimbabwe, approximately half (58.3%) of facilities had all 4 BLS items during the second year compared to 35.7% during the initial year (+22.6, 95% CI: −9.3, 54.5).
Similar patterns were observed for hygienic and safe infrastructure. Kenya maintained relatively high scores with no significant changes with 75.9% and 86.2% of facilities having all 4 items each year, respectively (+10.3, 95% CI: −9.7, 30.3). Tanzania maintained relatively low scores with 33.3% and 17.2% of facilities (−16.1, 95% CI: −46.1, 13.9). Higher scores in South Africa significantly declined from 53.3% to 12.5% of facilities (−40.8, 95% CI: −68.1, −13.6). Lower scores in Zimbabwe significantly improved from 28.6% to 83.3% (+54.7, 95% CI: 26.8, 82.8). Most facilities in Kenya, Tanzania, and Zimbabwe had high service availability, averaging 4.3 to 4.9 of the 5 package services in 2011, which were maintained during expansion. Significant service erosion was observed in South Africa from 4.8 to 3.5 of the 5 package services.
Table 2 shows item-level analyses of facility preparedness. Protocol availability varied among countries. Supplies most commonly unavailable were HIV post-exposure prophylaxis, followed by antibiotics and condoms. Absence of oxygen and anaphylaxis medications appeared to drive low BLS scores, while absence of adverse event monitoring systems and lighting drove lower infrastructure scores. With the exception of referral services, most VMMC facilities offered group education, individual counseling, and HIV counseling and testing.
Table 2. Item-Level Availability of VMMC Protocols, Equipment, Supplies, and Services from Initial Year (2011) to Expansion Year (2012).doi:10.1371/journal.pone.0079524.t002
Quality of VMMC Surgeries
Table 3 summarizes the quality of observed VMMC surgeries. Pre-operative tasks were correctly done in 79.5% and 97.1% of observed surgeries in Kenya and Zimbabwe, respectively, during the initial year and were maintained during the second year: (Kenya: −8.5, 95% CI: −17.6, 0.6 and Zimbabwe: +2.9, 95% CI: 0.6, 5.2). Pre-operative tasks were omitted the most in Tanzania's observed surgeries with low scores during the initial (20.6%) and second (26.3%) year (+5.7, 95% CI: −3.4, 14.8). Pre-operative tasks were correctly done in half (52.1%) of observed surgeries in South Africa, although this significantly declined over time (−39.8, 95% CI: −47.7, −32.0).
Table 3. Summary of Quality of VMMC Surgical Care from Initial Year (2011) to Expansion Year (2012).doi:10.1371/journal.pone.0079524.t003
Quality scores were highest during the operative period for surgical techniques. On average, 11.5 to 12.5 of 13 tasks were correctly done across countries. High scores were maintained during the follow-up year in Kenya (−0.1, 95% CI −0.3, 0.01) with significant improvements in Zimbabwe to 12.9 (+1.4, 95% CI 1.1, 1.7). Scores significantly decreased in South Africa to 11.6 (−0.8, 95% CI: −1.1,−0.5) and in Tanzania to 12.0 (−0.5, 95% CI: −0.7, −0.3). Scores for specific surgical tasks are shown in Table 4. Except for Zimbabwe, tying the surgical knot was consistently done incorrectly across countries, resulting in lower percentages of observed surgeries during expansion for which all 13 operative tasks were done correctly (Kenya 50.2%, South Africa 33.7%, Tanzania 36.7%, Zimbabwe 94.1%). Other incorrectly performed operative tasks were identification of the skin to be excised, administration of local anesthesia, and securing mattress sutures.
Table 4. Item-Level Quality of VMMC Surgical Care from Initial Year (2011) to Expansion Year (2012).doi:10.1371/journal.pone.0079524.t004
While on average the majority of safe operative procedures were correctly done during the initial year (6.3 to 6.9 of the 8 minimum tasks), protective eyewear was often omitted in all 4 countries resulting in few surgeries where all 8 safety tasks were completed (Kenya: 4.1%, South Africa, 9.7%, Tanzania 9.8%, Zimbabwe 0%). Kenya maintained otherwise high safety scores (−0.1, 95% CI: −0.2, 0.04), with significant improvements in Tanzania (+0.5, 95% CI: 0.3, 0.7) and Zimbabwe (+0.5, 95% CI: 0.4, 0.6). Significantly fewer safety tasks were done in South Africa's expansion year as compared to the initial year (−1.0, 95% CI: −1.3, −0.7). Lower safety scores reflected lack of hand washing, not disinfecting hands between clients, or not maintaining sterile surgical fields.
Quality scores were lowest during the post-operative surgical period with significant decreases in three countries: Kenya, 4.4 to 3.8 out of 7 items (−0.6, 95% CI: −1.0, −0.2); South Africa, 4.0 to 2.4 (−1.6, 95% CI: −2.0, −1.3); and Tanzania, 4.5 to 3.9 (−0.6, 95% CI: −1.0, −0.2). Providers in Zimbabwe significantly increased the average number of post-operative tasks correctly done from 4.9 to 6.1 (+1.2, 95% CI: 1.1, 1.3). In the second year, only a small proportion of surgeries had all 7 post-operative tasks performed (Kenya 18.9%, South Africa 0.6%, Tanzania 8.5%, Zimbabwe 9.8%). Post-operative tasks usually omitted were monitoring patients for reactions and reviewing vital signs, as well as providing post-operative counseling and reminding patients of the post-operative abstinence period.
Scaling up VMMC programs is an important strategy in the prevention of HIV infection in sub-Saharan African countries. Previous VMMC quality assessments have often focused on reporting of adverse events. However, no research to-date has examined changes in facility preparedness and correct performance of VMMC procedures as countries scale-up the number of facilities. Given the potentially significant contribution of VMMC in decreasing HIV transmission, monitoring quality is critical in identifying and eliminating gaps in care during introductory and scale-up periods.
This study's comparative process evaluation demonstrated that high quality VMMC services can be provided during rapid scale-up in sub-Saharan African settings. Nearly all VMMC operative tasks observed in the study were performed correctly, with the exception of tying surgical knots. However, the evaluation revealed several areas of sub-optimal VMMC performance levels. Lower scores in infection control during surgery and pre- and post-operative tasks highlighted the need to reinforce provider training and supervision in these areas. Pre-operative examinations are necessary to assess eligibility for surgery and identify surgical contradictions which may lead to adverse events. Post-operative tasks are essential in prevention, identification, and management of complications. Reinforcement of counseling messages during the post-operative period on the importance of risk reduction and sexual abstinence during wound-healing is also crucial. Lower scores among these items could influence the overall effectiveness of VMMC given that transmission rates are directly affected by these behaviors.
When examining overall scores, there were considerable variations across the two years of program implementation. Results showed that high levels of service quality can be maintained at-scale and, in some cases, dramatically improved during the process of scale-up, as was the experience of Kenya and Zimbabwe, respectively. However, analyses also underscored the challenge of replicating initial successes at-scale in the case of South Africa, and the challenge of managing concurrent performance improvement and deterioration within the VMMC program in Tanzania. Some safety measures improved, while surgical technique and post-operative care quality declined. During the second year in both countries, less than a third of sampled sites had all requisite protocols to equip staff with readily accessible information. Equally as few sites had all necessary equipment and supplies for normal and emergency surgeries. The implications for expanding VMMC services in South Africa are further detailed in this supplement . In all four countries, the absence of HIV post-exposure prophylaxis was more severe during the second year, and shortages at both time points of antibiotics and BLS equipment can hinder the management of severe adverse events.
Several lessons learned from this evaluation may inform future efforts to scale-up VMMC services in similar settings. One key finding is that while high quality VMMC programs can be achieved at-scale, the process of scaling-up does not guarantee compliance with all quality criteria. This was evidenced by the variability over time in facility preparedness and surgical care within countries. In some settings, quality measures declined over the two-year period, or the VMMC program was expanded in the second year with comparably low performance levels as observed during the first year. This evaluation emphasizes the need to routinely measure aspects of quality, and ensure that VMMC program expansion is linked with quality improvement initiatives.
Secondly, while the diversity of program experiences among countries precludes any definitive comparison, some insight may be drawn from the range of scale-up models which were used. All countries partnered with implementing agencies with variations in the implementation of fixed, mobile, and outreach sites. In addition, focused models provided the advantage of allocating resources to regions with greatest disparities in HIV prevalence and male circumcision, while national models benefited from population-wide implementation. Zimbabwe additionally delivered its VMMC program through a single implementing organization. There were also differences across countries in the volume of VMMC facilities added during the evaluation's two-year period. A final lesson emerging from this analysis is the recognition that challenges in commodity procurement and provider capacity may have reflected broader health systems strengthening needs that were not unique to VMMC. Careful attention will be needed to ensure that the urgency of scaling up VMMC does not bypass efforts to strengthen health systems or hinder quality of existing health services in facilities where VMMC is introduced.
The study's limitations deserve mention. VMMC facility preparedness was measured at the time of the study visit and does not reflect cumulative availability or adequacy of supplies relative to client load. Thus, use of point-in-time measures limited the evaluation's ability to examine seasonal trends in VMMC quality as the program expanded. Data were also collected during low volume service periods which may have confounded the study's observations, and the small number of facilities may have contributed to the wide confidence intervals of some measures. In addition, it is possible that in some instances the evaluation of VMMC surgeries was biased given that scores were based on the evaluator's interpretation. To minimize this, all observers were trained using a standardized observation guide. However, it was not possible to standardize ratings across countries which may have resulted in unequal scoring.
The evaluation's summary measures should likewise be interpreted carefully. Quality of VMMC services was examined from the perspective of the health system. Client perspectives of quality such as comfort, privacy, or communication adequacy are not reflected, and other systems-level quality metrics not measured in this study may have been informative. Furthermore, it can be argued that not all items were equally critical, and thus the analysis' equal weighting of items unduly magnified (or diluted) the clinical relevance of certain tasks. Although the development of differential weights was not feasible within the study's parameters, to reduce this possibility, we aimed to balance the interpretive weight of each score by including in the questionnaire only the minimally essential elements in each category. Thus, the range of scores is designed to reflect only items necessary to perform safe and effective circumcision. Finally, the results from this study are unique to the selected countries and may not be generalizable to other settings. This analysis was not designed to identify factors associated with differences in observed quality measures. Therefore, causal or comparative inferences cannot be drawn between countries or facilities.
To our knowledge, this study is the first to examine facility preparedness and quality of VMMC services in contexts of scale-up in four African countries. Lapses in high performance and, in some cases, expansion of considerably deficient services emphasize the need to better integrate VMMC scale-up and quality improvement processes. Monitoring service quality, not just adverse event reporting, will be essential in realizing the full health impact of VMMC for HIV prevention in African settings.
The authors wish to acknowledge the organizations that conducted the research in each country: FHI360/Kenya, CHAPS in South Africa, MCHIP Jhpiego/Tanzania, and PSI/Zimbabwe in collaboration with ZAPP-UZ. Co-investigators included Dr. Nicholas Muraguri, Dr. Peter Cherutich, Dr. Kawango Agot, Dr. Walter Obiero and Dr. Jackson Kioko (Kenya); Dr. Dirk Taljaard Dr. James McIntyre (South Africa), Dr. Bennet Fimbo and Dr. Eleuter Samky (Tanzania), and Dr. Karin Hatzold and Mr. Christopher Samkange (Zimbabwe). Participating in the data collection were Nicolas Pule, Mulashi Biola, Daniel Shabangu, Sindiswe Zwane, Sindiswe Maseko, Dr. Sifuni Koshuma, Milton Kabiligi, Dr. Tendai Mutwirah, Dr. Eric Nyazika, Dr. Kelvin Nemayire, Rosemary Owigar, and Dr. Violet Naanyu. We thank the data managers: Flora Hezwa, Dr. Obadia Venance, Nyongole, Dudzai Mureyi. Kate Macintyre assisted on sampling (Kenya) and Hally Mahler provided oversight (Tanzania). Emmanuel Njeuhmeli, Delivette Castor, and Jason Reed were co-investigators on SYMMACS. Dr. Arti Shankar at Tulane SPHTM provided statistical consultation. Drs. Deanna Kerrigan and Caitlin Kennedy of Johns Hopkins Bloomberg School of Public Health provided technical guidance over the life of the project. Andrea Vazzano and Emily Hurley provided editorial assistance for the final manuscripts.
Conceived and designed the experiments: DR JB EN CS BF PC DC . Performed the experiments: DR JB LP DA. Analyzed the data: LJ DA DR JB LP DC EN. Contributed reagents/materials/analysis tools: LJ JB DR SH KH CS DA BF PC LP DC EN. Wrote the paper: LJ JB DR SH KH CS DA BF PC LP DC EN.
- 1. Auvert B, Taljaard D, Lagarde E, Sobngwi-Tambekou J, Sitta R, et al. (2005) Randomized, controlled intervention trial of male circumcision for reduction of HIV infection risk: the ANRS 1265 trial. PLoS Med 2: e298.
- 2. Bailey RC, Moses S, Parker CB, Agot K, Maclean I, et al. (2007) Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomized controlled trial. Lancet 369: 643–56.
- 3. Gray RH, Kigozi G, Serwadda D, Makumbi F, Watya S, et al. (2007) Male circumcision for HIV prevention in men in Rakai, Uganda: a randomized trial. Lancet 369: 657–66.
- 4. Mwandi Z, Murphy A, Reed J, Chesang K, Njeuhmeli E, et al. (2011) Voluntary medical male circumcision: translating research into the rapid expansion of services in Kenya, 2008–2011. PLoS Med 8 (11) e1001130.
- 5. Dickson K, Tran N, Samuelson J, Njeuhmeli E, Cherutich P, et al. (2011) Voluntary medical male circumcision: a framework analysis of policy and program implementation in Eastern and Southern Africa. PLoS Med 8 (11) e1001133.
- 6. Edgil D, Stankard P, Forsythe S, Rech D, Chrouser K, et al. (2011) Voluntary medical male circumcision: logistics, commodities, and waste management requirements for scale-up of services. PLoS Med 8 (11) e1001128.
- 7. Njeuhmeli E, Forsythe S, Reed J, Opuni M, Bollinger L, et al. (2011) Voluntary medical male circumcision: modeling the impact and cost of expanding male circumcision for HIV prevention in eastern and southern Africa. PLoS Med 8 (11) e1001132.
- 8. Herman-Roloff A, Bailey RC, Agot K (2012) Factors associated with the safety of voluntary medical male circumcision in Nyanza Province, Kenya. Bull World Health Organ 90 (10) 773–81.
- 9. Mahler HR, Kileo B, Curran K, Plotkin M, Adamu T, et al. (2011) Voluntary medical male circumcision: matching demand and supply with quality and efficiency in a high-volume campaign in Iringa Region, Tanzania. PLoS Med 8 (11) e1001131.
- 10. Musau P, Demirelli M, Muraguri N, Ndwiga F, Wainaina D, et al. (2011) The safety profile and acceptability of a disposable male circumcision device in Kenyan men undergoing voluntary medical male circumcision. J Urol 186 (5) 1923–7.
- 11. Strategies and approaches for male circumcision programming. WHO Meeting Report, 5–6 December 2006. Geneva, Switzerland: World Health Organization 2006. World Health Organization website. Available: http://whqlibdoc.who.int/publications/2007/9789241595865_eng.pdf. Accessed 2013 Jun 5.
- 12. WHO (2009) Male Circumcision Services: Quality Assessment Toolkit. Geneva, Switzerland: World Health Organization. World Health Organization website. Available: http://www.who.int/hiv/pub/malecircumcision/qa_toolkit/en/index.html. Accessed 2013 Oct 18.
- 13. Ngo TD, Obhai G (2012) Male circumcision uptake, postoperative complications, and satisfaction associated with mid-level providers in rural Kenya. HIV AIDS (Auckl) 4: 37–43.
- 14. Muula AS, Prozesky HW, Mataya RH, Ikechebelu JI (2007) Prevalence of complications of male circumcision in Anglophone Africa: a systematic review. BMC Urol 7: 4.
- 15. Bailey RC, Egesah O, Rosenberg S (2006) Male circumcision for HIV prevention: a prospective study of complications in clinical and traditional settings in Bungoma, Kenya. Bull World Health Organ 86: 669–677.
- 16. Ford N, Chu K, Mills EJ (2012) Safety of task-shifting for male medical circumcision: a systematic review and meta-analysis. AIDS 26 (5) 559–66.
- 17. Bertrand JT, Rech D, Aduda D, Frade S, Loolpapit M, et al.. (2013) Systematic Monitoring of the Voluntary Medical Male Circumcision Scale-up: adoption of efficiency elements in Kenya, South Africa, Tanzania, and Zimbabwe. In press.
- 18. Rech D, Spyrelis A, Frade S, Perry L, Farrell M, et al.. (2013) Implications of the fast-evolving scale-up of adult Voluntary Medical Male Circumcision for quality of services in South Africa. In press.