A health systems strengthening intervention to improve quality of care for sick and small newborn infants: results from an evaluation in district hospitals in KwaZulu-Natal, South Africa

The global burden of neonatal mortality is substantial. In 2016 an estimated 7000 newborn babies died each day, with newborn deaths accounting for 46% of all deaths among children under 5 years [1]. Poor intrapartum and newborn care is also associated with a major burden of disability [2]. As child mortality has improved in recent years, achieving improvements in neonatal mortality has fallen behind, as a result of which neonatal deaths make up an increasing proportion of all child deaths. Improving outcomes for newborn babies has, therefore, been identified as a priority for global child health [3]. Proven, cost-effective interventions exist to manage the major causes of deaths in newborns. Key actions which can be implemented at scale in resource constrained settings in order to prevent up to three million global newborn deaths are outlined in the WHO/UNICEF Every Newborn Action Plan [4].

Neonatal deaths occur predominantly in the first week of life, with nearly 50% of deaths occurring in the first 48 h. Effective interventions include improved delivery care, immediate care for the infant at delivery, preventive care for the healthy newborn, as well as care for sick and small newborns. Improving care for small and sick newborns has been neglected in the past, and it is estimated that this could prevent close to 600,000 newborn deaths globally every year, with most of this effect being achievable with district hospital care [5]. Care for small and sick newborns includes extra thermal care, support for feeding, antibiotics for infection, and kangaroo mother care. Improving quality of care for sick and small newborns requires health personnel that are trained and equipped to manage these babies. Inpatient care facilities play a crucial role for newborns requiring full supportive facility care [4]. In particular, all staff members with a role in caring for newborn babies must be trained and competent in newborn resuscitation, since up to 10% of newborns may require stimulation at birth and 5% require resuscitation at birth [6]. Adequate skills and facilities are needed at all levels of the health system if good quality newborn care is to be accessible for all babies.

Child mortality in South Africa (SA) remains unacceptably high with a large proportion of child deaths occurring in the neonatal period [7], so that improving outcomes for these infants is a priority. Between January 2012 and December 2013, more than 14,000 early neonatal deaths were recorded in the SA National Perinatal Problem Identification Programme (PPIP) database from 588 PPIP sites in the country [8]. Many births (46.5%) and neonatal deaths (42.3%) in SA occur in community health centres and district hospitals [8] where clinical services are provided by generalist doctors and nurses. As in other lower resource settings, many newborn deaths in SA are from potentially preventable causes, with the major causes of perinatal deaths being intrapartum birth asphyxia and prematurity. Preventable factors identified by the PPIP programme in 2012–13 included lack of equipment in the neonatal unit, inadequate neonatal management plans, and inadequate monitoring of babies’ condition [8]. All facilities where deliveries occur need appropriate resources and expertise to provide care for these infants, if deaths and disability are to be prevented. Improving newborn care, particularly in district hospitals, is key to the reduction of perinatal and neonatal mortality, and it is estimated that this could prevent thousands of infant deaths in SA.

This paper describes an evaluation of the quality of care provided at 39 district hospitals in KwaZulu-Natal (KZN) province over a three-year period during which a structured, multipronged initiative was undertaken to support newborn care in district hospitals. This initiative was known as the KwaZulu-Natal Initiative for Newborn Care (KINC).

An observational, cross-sectional facility survey was conducted in all 39 district hospitals in KZN at baseline, midpoint and endpoint of the 3 year intervention period, using the same data collection tools at each time point. An experienced professional nurse/midwife, who had attended training in the management of sick and small newborns (MSSN), conducted all data collection. At each time point, each facility was visited for a single day to collect data. Hospitals were informed of the date of the visit but were not provided with information about the assessment, and involvement of hospital staff during the visit was minimised. At midpoint and endpoint all staff on duty in the neonatal unit on the day of the visit were requested to complete a self-administered questionnaire to assess knowledge of KZN neonatal care guidelines.

Description of the KINC intervention

KINC was a multi-pronged health system strengthening intervention undertaken over a 3 year period (2013–2016), aimed at addressing challenges to provision of high quality newborn care in 39 district hospitals in KZN. A KINC task team was set up to oversee and plan project activities, led by the KZN Department of Health, and including key role-players from all levels of the health system and from all health districts. Ahead of KINC implementation, a two-day orientation workshop was conducted with managers responsible for management of neonatal nurseries in district hospitals to promote buy-in and ensure awareness and support for the initiative. The intervention consisted of implementation of the guidelines for management of small and sick newborn infants (MSSN) in district hospitals developed by the SA Department of Health [9]. MSSN implementation included: training for all cadres of health workers (HWs); development of training teams for MSSN in every district; skills development for health managers using an action learning methodology; and on-site mentoring visits to district hospitals (Fig. 1). A five-day MSSN training was conducted with doctors and nurses from all district hospitals, which included both theory and clinical practice and was conducted at the local referral hospital wherever possible, to build team work between HWs at district and referral hospitals. KINC training focussed primarily on developing knowledge and skills of HWs to manage sick and small newborns. Topics included: assess and classify; treat, observe and care (including maintaining normal body temperature, safe oxygen therapy, maintaining normal glucose, feeds and fluids; infection prevention and control; referral); assess feeding and counsel; follow up; and routine care for all newborns. Management of specific conditions of newborns included: apnoea/respiratory distress; preterm/low birth weight; acute infection; encephalopathy; seizures; jaundice; congenital abnormalities; syphilis; tuberculosis; and HIV. The training package consisted of newborn care chart books; training manuals; exercise manuals and facilitator’s manuals. Drug dosages, charts and recording forms were supplied in the training package. Participants were selected for training using routine DoH systems based on whether they were working in the neonatal nursery.

Mentoring visits were conducted by an advanced midwife, in partnership with district maternal child and women’s health (MCWH) supervisors, using a structured mentoring tool. Mentoring activities included assessment of: availability and functionality of essential newborn care equipment; availability of personnel in the neonatal nursery; compliance with infection prevention and control practises, and auditing of newborn records to assess compliance with MSSN guidelines. On completion of the visit challenges identified were presented to senior hospital and district management for their attention. Proposed actions were reviewed on subsequent visits.

On completion of the intervention, accreditation visits were conducted to all district hospitals by a team of senior clinicians and managers to assess the hospital’s performance, and to determine whether each hospital had achieved accreditation status for excellence in newborn care. Activities included in the accreditation assessment and accreditation outcomes for district hospitals, are described elsewhere [10].

Data was collected in all 39 hospitals at baseline (July–October 2013), and at the endpoint (January–April 2015) and in 38/39 hospitals at midpoint (October–December 2014). The neonatal unit at one district hospital was closed for renovations during the midpoint evaluation.

Average scores

Average scores for resources, care practices and resuscitation equipment from baseline to endpoint are shown in Tables 1, 2, 3. Of the 39 participating hospitals, 35 hospitals (89.7%) scored higher for resources, 33 (84.6%) hospitals scored higher for care practices, and 34 (87.2%) hospitals scored higher for resuscitation at endpoint compared to baseline. There was an overall increase of 62.9% in the average resources score, 26.2% improvement in care practices score, and a 49.1% improvement in resuscitation score from baseline to endpoint. Scores for the composite quality of care score, comprising all 88 items, improved from 42.0 (lowest- highest 29.0–64.0) at baseline to 60.7 (44.8–73.5) at endline, an increase of + 44.5%. There was a significant improvement in total mean scores between baseline and endline (p = 0.0012). Individual hospital composite scores at baseline and endpoint are shown in Fig. 2.

Knowledge of health workers

All health workers on duty on the day of the assessment completed a knowledge questionnaire at midpoint and endpoint. There were 106 health professionals from the 38 district hospitals working in neonatal units on the day of data collection at midpoint (range: 1 to 8 per facility), and 120 health workers in 39 hospitals at endpoint (range: 1 to 6 per facility).

Health workers who had received MSSN training had on average a higher knowledge score than health workers who had not been trained in newborn care over the project period. This was shown at midpoint and was sustained at end point although no further training was conducted between those time points (Fig. 3). Results of the knowledge scores did not contribute to the overall quality of care score.

Our data suggests that a multipronged intervention approach comprising training, mentoring, action learning and accreditation, implemented at scale, can lead to demonstrable improvements to many elements contributing to the quality of care provided for sick and small newborn infants, over a relatively short time period. Improvements were seen in the resources available in most district hospitals, including capital equipment, consumables and drugs, as well as functioning resuscitation equipment and deployment of staff. Such resources are the foundation of providing quality care, without which this cannot be achieved. In addition, care practices also improved, including staff knowledge, observed clinical practices, record keeping and audit practices. Improvements in knowledge scores suggest that HWs trained in newborn care during the project period had a better knowledge of care practices for sick and small newborns, although our methodology does not allow us to clearly infer that KINC training led directly to improved knowledge.

It has been frequently stated that training alone does not lead to sustained change in practice [11], our findings suggest that when combined with mentoring and accreditation, significant improvements can be achieved. This interlinked multipronged approach led to overall strengthening of the health system. For example, training provided neonatal unit staff with information about the newborn care policies that should be followed and why, what equipment was required, and how this equipment should be used. Although procurement of equipment and consumables was not a direct function of this project, during mentoring visits we encouraged health staff to obtain required equipment, and deploy it in the neonatal unit. Progress with deploying equipment was reviewed at subsequent mentoring visits. In this way, we were able to facilitate the transfer of knowledge and skills acquired during training to the workplace. Preparation for accreditation in the final year of the project further reinforced these messages, as hospital managers were informed of the standards against which they would be assessed. During accreditation, each hospital was visited and assessed by a group of senior managers and clinicians, this served to strongly motivate hospital managers to comply with recommendations for their hospital to achieve a good accreditation score [10].

Several challenges were experienced in project implementation including: being unable to train HWs together as hospital teams because of conflicting clinical commitments; patchy coverage of paediatric outreach services and poor buy-in from outreach paediatricians; difficulties with scheduling mentoring visits when all local and district roleplayers were available to participate. We were able to adapt implementation plans to effectively address most problems because of strong leadership from the DoH at provincial and district levels. The provincial KINC task team included role-players from all over the province and meetings were consistently well attended, this allowed child health managers to participate and engage in all decision-making about project activities.

However, key gaps remained, some aspects of infrastructure did not improve, which is to be expected since infrastructure is difficult to change in a short time and requires considerable resources. However, many neonatal units still did not have a designated doctor, aspects of routine postnatal care remained poor, and, despite improvements, essential resuscitation equipment was still not available in all areas at endpoint. The Department of Health failed to provide all the required equipment for CPAP, and as a result implementation of CPAP remained inadequate in most hospitals, despite this having been identified as a key national priority to improve mortality in district hospitals [12]. This highlights the particular complexity of improving care for sick newborn babies. In contrast to many other child survival interventions, significant and ongoing technical expertise and equipment are required to support newborn care. In settings where health workers are scarce and systems for procurement and equipment maintenance are challenging, improvements to resources may be difficult to sustain. We were unable to assess sustainability beyond the completion of the project.

Training in newborn care requires skilled and experienced facilitators, including paediatricians, to teach clinical skills, making ongoing training difficult to sustain in low resource settings. High rates of staff turnover are an additional challenge. A mentorship-based approach could provide an alternative to residential training, so that nurses providing clinical care for newborns in district hospitals could spend time in regional hospitals being mentored and building their skills. Ongoing outreach programmes have a role to play, and outreach paediatricians should view mentoring of neonatal unit staff as a core activity during their visits. Innovative solutions may be required to reduce staff turnover and attract staff to work in neonatal units. More task shifting to nurses, supported by increases in the scope of practice, could be way to achieve this and has been shown to improve care and improve the motivation and retention of nurses [13, 14]. Other approaches could include improved remuneration of nurses with special skills in newborn care, although such interventions must be implemented with care as they may have the unintended consequence of diverting staff from other key areas of practice. Such approaches are identified as applicable to address global challenges in improving motivation of staff and retaining staff working in newborn care [15].

The scoring system developed for this evaluation provided an objective approach to track changes over time in each hospital, compare hospitals at each time point, and identify common shortfalls to prioritise for intervention [16, 17]. However, defining and measuring quality of care is complex [18–20] and multi-dimensional, and different aims for measuring quality may be as diverse as cost containment and patient satisfaction [21]. The most important limitation to our approach was the difficulty in measuring the quality of clinical care, particularly adherence to evidence-based clinical guidelines, which is the most important aspect of providing good quality care. Availability of equipment, consumables and human resources, and even staff knowledge and skills, are relatively easy to measure but do not go far in determining whether the care for newborn infants has actually changed. Although we included a record review to assess quality of care, the complexity of neonatal care for babies whose clinical condition may be varied and unpredictable, made it difficult to determine whether ongoing care was provided according to the guidelines, As a result, care variables were limited to a small number that could be easily assessed, focussing mainly on record keeping and routine observations. We were unable to directly measure adherence to guidelines, this would have required a skilled clinician, which was not feasible for this study.

To address this concern we also considered a number of outcome measures as possible indicators of quality of care, including length of hospital stay, adherence to guidelines and overall in-hospital mortality. However, it was difficult to compare outcomes across different facilities because of the range of clinical conditions and complications that can arise, as well as the substantial differences in numbers of admissions and access to referral care among facilities. In addition, outcomes for newborns are influenced by factors not directly related to clinical newborn care, for example the mother’s socioeconomic situation. All of these factors made it difficult to develop valid and reliable tools to assess and compare clinical care across facilities. We, therefore, acknowledge that while the KINC approach clearly demonstrated improvements in many of the building blocks required for quality care provision, without which this cannot be achieved, we were unable to directly assess whether clinical care or health outcomes improved. However, process indicators have a place, and should remain central to any assessment of quality of care for several reasons: they are easier to measure on an ongoing basis; they allow direct comparison between facilities; they can be measured at a specific time point without having to wait for complicated analysis; and can quickly provide direction for action to address problems. In contrast, addressing poor outcomes requires going back to process indicators to try and explain the poor outcomes [22].

Another challenge was the difficulty of weighting variables to provide a score of quality of care that accurately reflects quality. While it is clear that not all variables are equally important, it was challenging to determine exactly how much more important one element of care was compared to another. We, therefore, chose to use a large number of equally weighted variables to give an overview of quality of care. However, it should be acknowledged that using this approach hospitals could receive good and improving scores, while still failing to comply with key indicators, giving a misleading impression of the quality of care provided. Alternative approaches could be to select several critical indicators and weight these within the scoring system, or penalise hospitals who fail to comply with them. Such an approach worked well in the accreditation process undertaken during this project which is described elsewhere [10].

A future scoring system for quality of care could include additional data elements to strengthen the assessment of quality of care, including a more comprehensive and structured skills assessment for staff, particularly of resuscitation skills in different sites where resuscitation may be required. Vignettes have been successfully used to assess clinical skills in newborn care practices [23], and would have strengthened the methodology in this study. Mothers of infants in the neonatal unit and postnatal ward could also be interviewed to evaluate satisfaction with the care that they and their infants received. A strength of this scoring system was that it was easy to administer, and although a more comprehensive record review and skills assessment would be valuable, this would require skilled assessors and may have compromised the validity and reliability of the tool.

In addition, there were further limitations to the methodology in this study. It is not possible to clearly infer that knowledge scores had increased as a direct result of the intervention because there are alternative explanations for this, including that more competent HWs were selected for training or that those who were trained were more likely to be retained in the nursery and gain further skills. Further, although the data collector was requested to randomly select participants for the skills assessment, this was not done consistently, and it is possible that more skilled HWs were selected for the skills assessment. Finally, the quality of care scores relied on observations and reported findings on a particular day, so that both reporting and observation bias may have led to higher scores being achieved. Overall, it is not possible to draw a clear inference that the improvements demonstrated are directly attributable to the implementation of the KINC programme. However, it was not feasible or acceptable to exclude facilities from the intervention to provide a comparison group, and there was no other initiative directly targeting quality of newborn care over the 3 year KINC implementation period. Although neonatal mortality was not assessed as part of the quality of care score, routine statistics on neonatal mortality in KZN show no trends towards improvement over the period of KINC implementation, but these data are difficult to interpret because of poor quality and incomplete data [24].

Overall, the key to achieving sustained improvement in newborn care is leadership at all levels of the health system, and holding local managers accountable if improvement are not made. This project was strongly supported by current DoH policies, senior managers at the DoH, and district level managers, and was guided by a task team comprising role-players at all levels of the health system, so that extensive buy-in and support was created for improving newborn care. We believe this was pivotal to achieving success.

In conclusion, combining skills development, with mentoring and accreditation can provide policy-makers with information about gaps in care, where to prioritise additional resources and directly improve quality of care, for newborn infants.