Effect of skill drills on neonatal ventilation performance in a simulated setting—observation study in Nepal

Aim Maintaining neonatal resuscitation skills among health workers in low-resource settings will require continuous quality improvement efforts. We aimed to evaluate the effect of skill drills and feedback on neonatal resuscitation and the optimal number of skill drills required to maintain the ventilation skill in a simulated setting. Methods An observational study was conducted for a period of 3 months in a referral hospital of Nepal. Sixty nursing staffs were trained on Helping Babies Breathe (HBB) 2.0 and daily skill drills using a high-fidelity manikin. The high-fidelity manikin had different clinical case scenarios and provided feedback as “well done” or “improvement required” based on the ventilation performance. Adequate ventilation was defined as bag-and-mask ventilation at the rate of 40–60 breaths per minute. The effective ventilation was defined as adequate ventilation with a “well done” feedback. We assessed the correlation of the number of skill drills and clinical case scenario with adequate ventilation rate using Pearson’s correlation. We assessed the correlation of the number of skill drills performed by each participant with effective ventilation using the Mann-Whitney test. Results Among the total of 60 nursing staffs, all of them were competent with an average score of 12.73 ± 1.09 out of 14 (p < 0.001) on bag-and-mask ventilation skill checklist. Among the trained staffs, 47 staffs participated in daily skill drills who performed a total of 331 skill drills and 68.9% of the ventilations were done adequately. Among the 47 nursing staffs who performed the skill drills, 228 (68.9%) drills were conducted at a ventilation rate of 40–60 breaths per minute. There was no correlation in the adequate ventilation with the skill drill category (p = 0.88) and the level of skill performed (p = 0.28). Out of 47 participants performing the skill drills, 74.5% of them had done effective ventilation with a mean average of 8 skill drills (SD ± 4.78) (p value 0.032). Conclusion In a simulated setting, participants who had an average skill drill of 8 in 3 months had effective ventilation. We demonstrated optimal skill drill sessions to maintain the neonatal resuscitation competency. Further evaluation will be required to validate the findings in a scale-up setting.

What is known of this subject -Quality improvement interventions include training, reminders, audit and feedback, improvement in skill, and maintaining the neonatal resuscitation skills -There is a subsequent decay in the neonatal resuscitation skills if continuous refresher trainings are not provided -Neonatal resuscitation training and retraining require resources What is new from this study -Self-training with a high-fidelity, feedback simulator can maintain the neonatal resuscitation skills -For effective ventilation, the ventilation rate should be maintained along with the automated and objective feedback from the high-fidelity simulator -At least 8 skill drills are optimal to maintain the neonatal resuscitation skill for a period of 3 months Background An estimated 5.5 million deaths take place every year after 28 weeks of pregnancy within the neonatal period. Of these deaths, 1.8 million deaths pertain to intrapartum [1,2]. The Lancet Global Health Commission on High Quality Health Systems in the Sustainable Development Goal Era estimates that almost two thirds of these deaths are associated with poor quality of care [3]. Every Newborn Action Plan-2014 aims to reduce the preventable stillbirth and neonatal death by 2030 in all income settings [4]. Reducing preventable death requires high-quality care and systems for managing mothers and newborns. Systematic review has shown that neonatal resuscitation, if delivered in a high-quality standard, can reduce intrapartum-related death by 30% [5]. Effective scale up of the neonatal resuscitation program is required to maintain the effect of the intervention on intrapartum-related death [6,7]. However, there are barriers at different levels of governance (macro, meso, and micro level) for an effective implementation of neonatal resuscitation [7]. Different quality improvement (QI) interventions are recommended at different levels of governance to overcome the barriers for effective implementation [8,9]. The QI interventions at a system level include governance and financing, at a hospital level involve setting accountability for improving care, and at a health worker level comprise training, reminders, audit, and dissemination of the guideline [9]. American Academy of Pediatrics developed a pictorialbased neonatal resuscitation program with training package called Helping Babies Breathe (HBB) for middle-and low-income settings in 2010 [10]. The objective of the program was to improve health worker's performance in simulated as well as clinical settings [11]. Evaluation of HBB program has shown improvements in the health workers' performance as well as mortality outcomes in low-and middle-income settings [12][13][14]. A systematic review of HBB implementation on birth outcomes showed 34% and 30% reduction in intrapartum stillbirth (RR 0.66; CI 95% 0.52-0.85) and first day mortality (RR 0.70; CI 95% 0.51-0.98), respectively [15].
Different quality improvement interventions-training, reminders, and audit and feedback have been used to implement the HBB program [16]. In a rural hospital in Tanzania, training and retraining were used as QI interventions for improving health workers' neonatal resuscitation performance in a simulated setting [17]. In a multi-country study conducted in India and Kenya, daily skill drills and reminders were used as QI interventions for maintaining neonatal resuscitation skills [18]. In a tertiary hospital study in Nepal, audit and feedback were used as QI interventions for reducing perinatal mortality using HBB [13].
Despite the description of the QI interventions in the HBB program, there are evidence gaps on QI interventions that improve and maintain neonatal resuscitation performance in simulated settings [19][20][21]. One of the most recommended QI interventions in resuscitation is daily skill drills [22]. Barriers exist in the implementation of daily skill drills in clinics for the health workers [23].
We aimed to evaluate the effect of skill drills and feedback on neonatal resuscitation performance and optimal skill drills required to maintain the ventilation skills in a simulated setting.

Methods
This study is reported as per strengthening the reporting of observational studies in epidemiology (STROBE checklist) [24].

Study design
This is an observational cohort study.

Study setting
We conducted this study in a referral level hospital of Nepal Pokhara Academy of Health Sciences, located in western Nepal. The 250-bedded hospital has an annual delivery rate of 8560 with stillbirth rate of 19 per 1000 births and pre-discharge neonatal mortality of 15 per 1000 live births [25].
The hospital had four units for managing mothers and newborns.
1. Labor unit with two delivery beds were managed by obstetricians and nurses. The normal and complicated vaginal deliveries took place in the labor unit. The labor unit had 15 nursing staffs.
2. Operating theater was managed by obstetricians and surgeons, where cesarean sections took place. There were two operating tables. There were a total of 15 nursing staffs. 3. Postnatal unit managed by obstetricians and nurses.
The mothers and newborns were kept in observation following low-risk delivery. There were 8 beds with a total of 15 nursing staffs. 4. Neonatal care unit was managed by pediatricians and nurses where the sick newborns were kept for treatment. There were 8 radiant warmers, 2 CPAP machine, and 1 phototherapy machine. There were a total of 15 nursing staffs.
The study was conducted from 15 July to 15 October 2018.

Sample size
We included all the nursing staff working at the maternal and newborn care units.

Participants
The nurses working in all four units were eligible as they have been working to provide care for mothers and babies at the time of birth and emergency condition. An orientation meeting was conducted with the nursing department on the objective of the study and nurses were consented. A total of 60 participants were enrolled in the study.

Interventions
The participants were given a 1-day training on the HBB second edition (2.0) package [26]. The HBB 2.0 followed the International Liaison Committee on Resuscitation (ILCOR) 2015 updated guideline for management of the non-breathing baby's basic neonatal resuscitation processes with the cord intact and less use of suctioning unless warranted [27]. HBB 2.0 is a skill-based training with practice on a manikin. The manikin (NeoNatalie) is a synthetic baby doll which is water filled and weighs 2.5 kg [28]. It has a plastic-foil lung in the chest which shows chest rise when ventilations are provided. The HBB training package also included upright bag-andmask ventilation, reusable suction in the shape of a penguin, two cord ties, two sets of towels, a baby cap, and an umbilical cord in the set. The training was organized in a batch of 20, arranged in a group of four in each table. Each table had two manikins such that the steps on neonatal resuscitation could be practiced in a pair for at least 40 times during the training session. On the first day, the standard training on HBB 2.0 was provided.
On the second day, the high-fidelity manikin (NeoNatalie Live) which simulates different clinical scenario was introduced to the training participants. NeoNatalie Live was developed to give a more life-like realism of the challenges that often occur in the resuscitation of newborn babies, to simulate a realistic lung and heart rate development after birth, and to measure and help improve the skills of personnel performing newborn resuscitation [28].
This high-fidelity manikin had sensors that measure resuscitation skills in the form of head positioning for keeping the airway open, mask seal, air pressure given to the lungs, ventilation rate, and continuity of ventilation. After a sufficient time of adequate ventilation (40-60 breaths per minute), the manikin cried as a signal of spontaneous breathing. The manikin had wireless connectivity to a tablet-based application (app) from which three patient cases (skill levels) could be selected, including different initial heart rates and the condition of the lungs. The levels were categorized as easy, medium, and hard. The "easy" level had a normal heart rate and open lungs. The "medium" level had a low heart rate and open lungs. The "hard" level had a low heart rate with closed lungs. Immediately after each resuscitation practice, the app provided feedback to the participants on how the resuscitation was performed and how to improve next time. If the app provided feedback as "well done," then the participant had done the clinical case scenario as per the ventilation step. If not, the app provided feedback on the ventilation step which required improvement. The app also recorded the names of the participants and created a participant ID linked to the resuscitation practice. All results were automatically uploaded to a cloud database (Microsoft Azure).
The ventilation practice was performed with a new and improved resuscitator for newborns (the Laerdal Upright with a PEEP bag and mask). Equipped with a novel PEEP valve, it adds a positive end-expiratory pressure in the lungs of newborns. The ergonomically improved upright design and new mask make it easier to maintain mask seal (Additional file 1).
Each participant in the different maternal and newborn care units were instructed to practice the neonatal resuscitation on the manikin before they started their clinical work in each unit every day. Nurses registered their name in the app and practiced skill drills on the manikin during the study period.

Data collection
The evaluation of HBB 2.0 was done using a knowledgebased questionnaire (pre and post), a skill-based checklist, and clinical case scenarios (Additional file 2). The questionnaire for knowledge assessment had 17 multiple-choice questions and 7 skill-based checklists for bag-and-mask ventilation evaluation. Two clinical scenario-based questionnaires-Objective Structured Clinical Evaluation (OSCE) A (13 steps) and OSCE B (23 steps)-from the standard HBB evaluation package were used. The background information of the study participants such as age, education, experience in delivery care, and resuscitation was assessed. Evaluation during the training was done in a paper-based format.
The level of skill drills for a particular day was randomly selected by an independent research officer and provided to the study participants. The number of skill drills performed by each participant was manually recorded by the research officer.
The app recorded information on study participants, level of skill drills performed, and performance feedback following completion of each drill. Four dedicated research officers were assigned to observe the activities.

Dependent variables
Adequate ventilation-ventilation at the rate of 40-60 breaths per minute Effective ventilation-adequate ventilation with a "well done" feedback in 75% of the skill drill attempts

Independent variables
Number of skill drills conducted The level of skill drills: easy, medium, and hard clinical case scenario

Background variables
The demographic characteristics of the study participants (age, education, experience in delivery, and neonatal resuscitation) Knowledge before and after the training Skill and clinical scenario competency following the training (OSCE A and B)

Statistical methods
For comparison of the knowledge of the study participants, a paired t-test was used, and mean difference was calculated. For skill competency of the study participants following training, a one-sample t-test was used, and mean difference was calculated. The number of skill drills was categorized into 1-5, 6-10, and ≥ 11 sessions.
We assessed the correlation of the number of skill drills and clinical case scenario (level of skill-easy, medium, and hard) with an adequate ventilation rate. For this, Pearson's correlation was used to analyze the correlation between each category of skill drills and the level of skill with an adequate ventilation rate (40-60 breaths per minute) on each drill. We assessed the correlation of the number of skill drills performed by each participant with effective ventilation (75% of the attempt having 40-60 breath ventilation per minute with well done feedback). For this, we used a non-parametric test (Mann-Whitney test) which provided the mean (with standard deviation) and median (with inter-quartile range) for effective and ineffective ventilation.

Ethical approval
The study was approved by the Institutional Review Committee of the hospital as well as the Ethical Review Board of the Nepal Health Research Council (reg. 95-2018).
There was an improvement in knowledge on HBB 2.0 by 1.77 (95% CI 1.24-2.30) times among the participants. All participants scored more than 80% in the bagand-mask skill check, average score of 12.73 out of 14 (p < 0.001). In the OSCE A and B, the post-training score was more than 80% of the provided checklist. In the ventilation graph, the average ventilation rate among the 60 participants was 40.55 ± 2.66 (Table 2).
Among the total participants, 47 participated in the skill drills and a total of 331 drills were conducted. Due to the transfer of 13 study participants to another department, the lost to follow-up took place (Fig. 1). Among these transferred 13 nurses, 10 did not attend any births per month while 2 attended 1 to 5 births and 1 attended more than 50 births per month. Regarding resuscitation, 5 of these nurses had used a bag and mask to resuscitate a baby while 8 had not. Six of them were staff nurses while 7 of them were nurses with a qualification of Bachelor's degree and above. Among the total of 331 skill drills, 228 (68.9%) drills were conducted at a ventilation rate of 40-60 breaths per minute (adequate ventilation rate). Among those who performed 1-5 sessions, 70.4% of them were adequate; among those who performed 6-10 sessions, 65.7% of them were adequate; and among those who performed 11 sessions or more, 70.2% were adequate. There was no correlation in the adequate ventilation with the skill drill category (p = 0.88) and the level of skill performed (p = 0.28) ( Table 3).
Out of 47 participants performing the skill drills, 74.5% of them had done effective ventilation with a mean average of 8 skill drills (SD ± 4.78). The remaining 25.5% of them had not done effective ventilation. The mean average skill drills conducted by these participants were 4 skill drills (SD ± 2.61). With the non-parametric test (Mann-Whitney), we found correlation in the number of drills performed by each participant with effective  ventilation. An average of 8 drills per participant was associated with effective ventilation in comparison with effective ventilation (p value 0.032) ( Table 4).

Discussion
This evaluation suggests that study participants who had average skill drills of 8 in 3 months had effective ventilation (ventilating at the rate of 40-60 breaths per minute). At least 8 skill drills are required to maintain the skills for a period of 3 months for neonatal resuscitation. The number of drills and the level of skill performed have no correlation with the adequacy of ventilation.
There are several limitations in this study. First, this was an observational study, so equal proportion of drills for each participant would have improved the quality of evidence. Second, this study is done over a period of 3 months, so the waning effect of neonatal resuscitation skills between the drills over a period of time cannot be estimated. Third, there is an observation bias, since the research officers observed the drills.
A study in the USA on neonatal resuscitation skill decay following training and with retraining at different time points showed that the skill retained until 2 months after the last training but declined after 4 months [29]. Similar findings were seen in the implementation of HBB in Ghana, where the skill improved following refresher training at 4 months [19]. If the refresher training is provided at periodic intervals, the retention of the pediatric and neonatal intubation skills after simulationbased training is high [30]. Simulation, as a method in the neonatal resuscitation training, improves confidence, knowledge, and performance over time in comparison with lectures [31]. However, the skills decay in the simulation group at 6 months [32]. A randomized controlled trial on neonatal resuscitation program on the manikin simulator showed better confidence and satisfaction with a high-fidelity manikin [32]. There is a knowledge gap on the optimal dose of the daily skill drills in a high-fidelity simulator for retaining the neonatal resuscitation skills and some studies have demonstrated that simulated drills have an effect on neonatal resuscitation [18,33,34]. Based on those studies, HBB 2.0 recommends daily drills; however, this recommendation requires better evidence for implementation and advocacy.

Conclusion
Further reduction in intrapartum-related death requires effective implementation of the neonatal resuscitation guideline. The 2015 ILCOR education for improving good clinical practice recommends use of high-fidelity manikin, users' feedback, and periodic training [35]. We demonstrated the optimal dose of simulated drills and feedback required to maintain the neonatal resuscitation skills in simulated settings. Further, evaluation of the simulated drills and feedback in a scale-up setting will provide evidence for generalizability of the result. Further evidence is required, such as optimal dose of skill drills, to have an effect of neonatal resuscitation in a clinical setting.
Additional file 1. Pictorial form of simulated drills.
Additional file 2. Quality Improvement tool. Authors' contributions AKC, AG, and RG conceptualized the study. AKC did the data analysis and prepared the first draft of the manuscript. AKS did the second revision of data analysis and prepared the second draft of the manuscript. AG, RG, HM, OB, SG, ØHG, and SKS reviewed and provided their feedback. ØHG and HM participated in the development of the manikin and the software used in the study and provided technical support. All authors agreed to the final manuscript.

Funding
Innovation Norway and Laerdal Global Health provided funding for the project. The funding bodies had no role in the development of the study

Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate
The ethical approval for the study was taken from the ethical review board of Nepal Health Research Council (reg. no. 95-2018). Written consent was obtained from the participants enrolled in the study.

Consent for publication
Not applicable.

Competing interests
Øystein H. Gomo (ØHG), Helge Myklebust, and Sakina Girnary are employed by Laerdal Medical, a sister company of Laerdal Global Health. ØHG contributed to the overall study design but had no role in the interpretation of the data or in the decision to submit the paper for publication.