Our data suggests that a multipronged intervention approach comprising training, mentoring, action learning and accreditation, implemented at scale, can lead to demonstrable improvements to many elements contributing to the quality of care provided for sick and small newborn infants, over a relatively short time period. Improvements were seen in the resources available in most district hospitals, including capital equipment, consumables and drugs, as well as functioning resuscitation equipment and deployment of staff. Such resources are the foundation of providing quality care, without which this cannot be achieved. In addition, care practices also improved, including staff knowledge, observed clinical practices, record keeping and audit practices. Improvements in knowledge scores suggest that HWs trained in newborn care during the project period had a better knowledge of care practices for sick and small newborns, although our methodology does not allow us to clearly infer that KINC training led directly to improved knowledge.
It has been frequently stated that training alone does not lead to sustained change in practice [11], our findings suggest that when combined with mentoring and accreditation, significant improvements can be achieved. This interlinked multipronged approach led to overall strengthening of the health system. For example, training provided neonatal unit staff with information about the newborn care policies that should be followed and why, what equipment was required, and how this equipment should be used. Although procurement of equipment and consumables was not a direct function of this project, during mentoring visits we encouraged health staff to obtain required equipment, and deploy it in the neonatal unit. Progress with deploying equipment was reviewed at subsequent mentoring visits. In this way, we were able to facilitate the transfer of knowledge and skills acquired during training to the workplace. Preparation for accreditation in the final year of the project further reinforced these messages, as hospital managers were informed of the standards against which they would be assessed. During accreditation, each hospital was visited and assessed by a group of senior managers and clinicians, this served to strongly motivate hospital managers to comply with recommendations for their hospital to achieve a good accreditation score [10].
Several challenges were experienced in project implementation including: being unable to train HWs together as hospital teams because of conflicting clinical commitments; patchy coverage of paediatric outreach services and poor buy-in from outreach paediatricians; difficulties with scheduling mentoring visits when all local and district roleplayers were available to participate. We were able to adapt implementation plans to effectively address most problems because of strong leadership from the DoH at provincial and district levels. The provincial KINC task team included role-players from all over the province and meetings were consistently well attended, this allowed child health managers to participate and engage in all decision-making about project activities.
However, key gaps remained, some aspects of infrastructure did not improve, which is to be expected since infrastructure is difficult to change in a short time and requires considerable resources. However, many neonatal units still did not have a designated doctor, aspects of routine postnatal care remained poor, and, despite improvements, essential resuscitation equipment was still not available in all areas at endpoint. The Department of Health failed to provide all the required equipment for CPAP, and as a result implementation of CPAP remained inadequate in most hospitals, despite this having been identified as a key national priority to improve mortality in district hospitals [12]. This highlights the particular complexity of improving care for sick newborn babies. In contrast to many other child survival interventions, significant and ongoing technical expertise and equipment are required to support newborn care. In settings where health workers are scarce and systems for procurement and equipment maintenance are challenging, improvements to resources may be difficult to sustain. We were unable to assess sustainability beyond the completion of the project.
Training in newborn care requires skilled and experienced facilitators, including paediatricians, to teach clinical skills, making ongoing training difficult to sustain in low resource settings. High rates of staff turnover are an additional challenge. A mentorship-based approach could provide an alternative to residential training, so that nurses providing clinical care for newborns in district hospitals could spend time in regional hospitals being mentored and building their skills. Ongoing outreach programmes have a role to play, and outreach paediatricians should view mentoring of neonatal unit staff as a core activity during their visits. Innovative solutions may be required to reduce staff turnover and attract staff to work in neonatal units. More task shifting to nurses, supported by increases in the scope of practice, could be way to achieve this and has been shown to improve care and improve the motivation and retention of nurses [13, 14]. Other approaches could include improved remuneration of nurses with special skills in newborn care, although such interventions must be implemented with care as they may have the unintended consequence of diverting staff from other key areas of practice. Such approaches are identified as applicable to address global challenges in improving motivation of staff and retaining staff working in newborn care [15].
The scoring system developed for this evaluation provided an objective approach to track changes over time in each hospital, compare hospitals at each time point, and identify common shortfalls to prioritise for intervention [16, 17]. However, defining and measuring quality of care is complex [18,19,20] and multi-dimensional, and different aims for measuring quality may be as diverse as cost containment and patient satisfaction [21]. The most important limitation to our approach was the difficulty in measuring the quality of clinical care, particularly adherence to evidence-based clinical guidelines, which is the most important aspect of providing good quality care. Availability of equipment, consumables and human resources, and even staff knowledge and skills, are relatively easy to measure but do not go far in determining whether the care for newborn infants has actually changed. Although we included a record review to assess quality of care, the complexity of neonatal care for babies whose clinical condition may be varied and unpredictable, made it difficult to determine whether ongoing care was provided according to the guidelines, As a result, care variables were limited to a small number that could be easily assessed, focussing mainly on record keeping and routine observations. We were unable to directly measure adherence to guidelines, this would have required a skilled clinician, which was not feasible for this study.
To address this concern we also considered a number of outcome measures as possible indicators of quality of care, including length of hospital stay, adherence to guidelines and overall in-hospital mortality. However, it was difficult to compare outcomes across different facilities because of the range of clinical conditions and complications that can arise, as well as the substantial differences in numbers of admissions and access to referral care among facilities. In addition, outcomes for newborns are influenced by factors not directly related to clinical newborn care, for example the mother’s socioeconomic situation. All of these factors made it difficult to develop valid and reliable tools to assess and compare clinical care across facilities. We, therefore, acknowledge that while the KINC approach clearly demonstrated improvements in many of the building blocks required for quality care provision, without which this cannot be achieved, we were unable to directly assess whether clinical care or health outcomes improved. However, process indicators have a place, and should remain central to any assessment of quality of care for several reasons: they are easier to measure on an ongoing basis; they allow direct comparison between facilities; they can be measured at a specific time point without having to wait for complicated analysis; and can quickly provide direction for action to address problems. In contrast, addressing poor outcomes requires going back to process indicators to try and explain the poor outcomes [22].
Another challenge was the difficulty of weighting variables to provide a score of quality of care that accurately reflects quality. While it is clear that not all variables are equally important, it was challenging to determine exactly how much more important one element of care was compared to another. We, therefore, chose to use a large number of equally weighted variables to give an overview of quality of care. However, it should be acknowledged that using this approach hospitals could receive good and improving scores, while still failing to comply with key indicators, giving a misleading impression of the quality of care provided. Alternative approaches could be to select several critical indicators and weight these within the scoring system, or penalise hospitals who fail to comply with them. Such an approach worked well in the accreditation process undertaken during this project which is described elsewhere [10].
A future scoring system for quality of care could include additional data elements to strengthen the assessment of quality of care, including a more comprehensive and structured skills assessment for staff, particularly of resuscitation skills in different sites where resuscitation may be required. Vignettes have been successfully used to assess clinical skills in newborn care practices [23], and would have strengthened the methodology in this study. Mothers of infants in the neonatal unit and postnatal ward could also be interviewed to evaluate satisfaction with the care that they and their infants received. A strength of this scoring system was that it was easy to administer, and although a more comprehensive record review and skills assessment would be valuable, this would require skilled assessors and may have compromised the validity and reliability of the tool.
In addition, there were further limitations to the methodology in this study. It is not possible to clearly infer that knowledge scores had increased as a direct result of the intervention because there are alternative explanations for this, including that more competent HWs were selected for training or that those who were trained were more likely to be retained in the nursery and gain further skills. Further, although the data collector was requested to randomly select participants for the skills assessment, this was not done consistently, and it is possible that more skilled HWs were selected for the skills assessment. Finally, the quality of care scores relied on observations and reported findings on a particular day, so that both reporting and observation bias may have led to higher scores being achieved. Overall, it is not possible to draw a clear inference that the improvements demonstrated are directly attributable to the implementation of the KINC programme. However, it was not feasible or acceptable to exclude facilities from the intervention to provide a comparison group, and there was no other initiative directly targeting quality of newborn care over the 3 year KINC implementation period. Although neonatal mortality was not assessed as part of the quality of care score, routine statistics on neonatal mortality in KZN show no trends towards improvement over the period of KINC implementation, but these data are difficult to interpret because of poor quality and incomplete data [24].
Overall, the key to achieving sustained improvement in newborn care is leadership at all levels of the health system, and holding local managers accountable if improvement are not made. This project was strongly supported by current DoH policies, senior managers at the DoH, and district level managers, and was guided by a task team comprising role-players at all levels of the health system, so that extensive buy-in and support was created for improving newborn care. We believe this was pivotal to achieving success.
