Aims
Within the WMG study, our objectives are to:
-
1.
examine barriers and enablers for universal developmental surveillance in NSW from the perspective of policies, systems and processes using focus groups and in-depth interviews of stakeholders.
-
2.
assess risk factors for non-completion of 6-, 12-, and 18-month developmental surveillance from the perspective of parent participation and engagement;
-
3.
determine the prevalence and psychosocial correlates of developmental risk at these ages; and
-
4.
ascertain the diagnostic test accuracy of the current NSW universal developmental surveillance program and whether the addition of an autism specific screening tool at 18 months increases diagnostic accuracy.
The WMG study is a mixed methods study that includes both quantitative and qualitative components. The qualitative component is designed to evaluate the feasibility, and barriers and enablers of the current universal surveillance program in NSW (objective 1), whereas the quantitative component seeks to provide data addressing objectives 2, 3 and 4.
Setting
This study is being conducted in south western Sydney, an area of significant social disadvantage about 40 km from the Sydney central business district. The South Western Sydney Local Health District (SWSLHD) provides tertiary (including neonatal intensive care, inpatient and outpatient paediatric care) and community health services to the residents of this region. Children are referred to the Sydney Children’s Hospital Network for subspecialist paediatric services. SWSLHD is the largest health service in NSW and comprises seven local government areas. In 2011, the population of SWSLHD was estimated at 875,384 persons (11.7% of the NSW population), and is projected to increase by 18,000 people per annum over the next decade. By 2016, the population is expected to reach 958,397 people and 1.06 million by 2021. There were 12,997 births in south western Sydney in 2012, representing about 13 per cent of all births in NSW [30]. The total fertility rate ranges from 1.90 to 2.34 infants per woman compared to the NSW rate of 1.97 infants per woman [31]. This is a fast growing population including a large indigenous and culturally and linguistically diverse (CALD) community, characterised as having high unemployment, and the accompanying health and psychosocial concerns of disadvantaged populations [31].
Study design
Objective 1: qualitative study
Participants in this component of the study will comprise the main stakeholders, namely parents, community nurses, GPs, paediatricians, general practice nurses, pharmacy nurses, pre-school and day-care staff. We will conduct focus groups as well as in-depth individual interviews with participants to identify barriers to the universal developmental surveillance program, community’s awareness, accessibility and engagement with the surveillance program as well as the factors that might enable universal developmental surveillance. Parental knowledge about typical/atypical early childhood development and professional knowledge about developmental surveillance systems will also be ascertained. Participants from a range of socioeconomic and CALD backgrounds will be included in order to ensure that we fully capture the experiences including barriers and enablers to using the health services of families from a wide range of ethnic and language backgrounds. We will interview families from the main multicultural groups resident in south western Sydney, and use translations of study information and interpreters during this key stage.
The Grounded Theory Method will guide the interpretation and thematic analyses of these qualitative data as well as feed into hypotheses of later stages of the study [32].
Objectives 2 and 3: longitudinal study
A cohort of children will be recruited in three main study arms (see Figure 1). A birth cohort from two teaching hospitals in SWSLHD will form the primary target sample (birth arm). An additional prospective cohort will be recruited through community nurses in the community as this is the pathway being used currently in NSW for developmental surveillance (community nurse arm). These two study arms will be combined into the main prospective cohort of the study (prospective cohort). In addition, a retrospective arm of the study will recruit infants born in the two teaching hospitals that are currently in the age range of 18 to 21 months (retrospective cohort) and will serve as a comparison group.
For the birth arm, the WMG research team will attend the postnatal wards on a daily basis to recruit mothers who have given birth in the previous 24 hours at the two teaching hospitals in SWSLHD. The nursing staff on the postnatal wards will be consulted and their support sought for recruitment into the study. Recruitment information will be made available in English, Arabic, Vietnamese, Khmer, and Chinese - the predominant community languages used in south western Sydney. Families who agree to participate in the study will be asked to complete a Newborn Baseline Questionnaire (NBQ) which includes baseline sociodemographic and health service-use information, and will be informed about the subsequent follow-up telephone interviews when their infant is 6, 12, and 18 months of age.
For recruitment to the community nurse arm, parents will be informed about the study by community nurses during the Universal Health Home Visit that occurs by four weeks post-birth. If the family shows interest in learning more about the study, the contact information of parents who express interest will be provided to the WMG study research team. Researchers will then contact these parents and mail study information and consent forms with reply-paid envelopes to be returned.
The retrospective arm of the study will comprise infants born in the two teaching hospitals who are in the age range of 18 to 21 months at the time of recruitment. This sample of children will serve as a ‘surveillance as usual in the community’ comparison group (i.e., a control group). This will help to determine any participation bias by ensuring that the WMG study is not influencing the help seeking and health related behaviours of study participants via a Hawthorne effect. For example, it is possible that the awareness gained through participation in the study might make the families more compliant with the developmental surveillance programs being assessed.
The total target sample for the study from all three arms is expected to be 2000 children.
Follow-up of prospective cohort
Telephone follow-up interviews for the prospective cohort (i.e. those recruited through the birth and community nurse arms of the study) will consist of a 10-minute interview that will ask about their use of universal developmental screening services using a purposively developed Prospective Developmental Surveillance Questionnaire (PDSQ). These telephone interviews will occur after the infants reach 6, 12, and 18 months of age. The key questions will include whether the child has been taken for the recommended scheduled checks, which health service(s) has been used, satisfaction level with the service, whether a PEDS had been completed, and by whom. The researcher contacting the family for the follow-up calls will complete the PEDS with any parent who has not completed a PEDS as part their child’s personal health record developmental surveillance schedule. At the 18-month telephone call, components of the NBQ will be re-administered so that it is possible to compare the sociodemographic characteristics at the child’s birth and 18 months later, as well as further questions on social capital, access to early childhood education services, and parenting. The same procedures will be followed for the retrospective cohort, but all the data will be collected in one telephone call.
Underlying theoretical model and data analysis for longitudinal study
In order to plan comprehensive service models from the findings of the longitudinal study, we need to examine the complex transactional relationship between the child’s environment and biology over time [33]. We have developed a composite of a bio-ecological and life course model to serve as a framework within which to do this [34–37]. These theoretical models will be linked to appropriate analytical models and in this regard multilevel modelling will be employed to examine the independent impacts of community variables (e.g., socio-economic status), parental/family variables (e.g., family health, substance use, mental health history, country of birth, language spoken at home, parental perceptions about preventive healthcare and access to childcare and preschools), and child variables (e.g., temperament, preterm/low birth weight, intrauterine exposure to drugs, medicines, infections, low Apgar scores, perinatal complications, presence of developmental and behavioural concerns) on developmental risk and access to services. A combined risk index will be created that includes measures of biological, psychosocial, developmental and socioeconomic risk used in the previous multivariate analyses. Cumulative risk analysis will be undertaken to determine the relationship between burden of risk and uptake of referral recommendations and utilisation of services. In addition, we will examine the transactional impacts of these variables on the uptake of referral recommendations provided by the research team following the reference standard assessment and utilisation of health services.
Objective 4: diagnostic test accuracy study
All children who are identified as being at risk of having a developmental disorder using the PEDS (both ASQ positive and ASQ negative), will be invited to participate in a reference standard assessment (defined below) between 18 and 21 months of age. Of the approximately 2000 children that will be recruited to the study, it is estimated that a concern on the PEDS will be reported for around 800 children (40%) and of these approximately half (400 children) will be at a level of risk that warrants further assessment using the ASQ and the ASQ:SE. From the remaining 1200 PEDS negative children, every 12th child (n = 100) will also be invited to participate in all the additional assessments (ASQ, ASQ:SE, Modified Checklist for Autism in Toddlers, Mullen Scales of Early Learning, ADOS). Thus, it is planned that a total of 500 children will receive further reference standard assessments as described below.
The reference standard assessments will be conducted to assess whether the current program is accurately identifying children with ASD, global developmental delay, physical developmental problems, and speech and language concerns. We will also evaluate whether the addition of the Modified Checklist for Autism in Toddlers (M-CHAT) assessment [38] would improve correct identification of children with ASD. The parents will also complete the Short Temperament Scale for Toddlers [39]. The M-CHAT, ASQ, and ASQ:SE will be completed by the parents prior to the reference standard assessment but the researchers carrying out the assessments will be blind to the child’s scores on PEDS, M-CHAT, ASQ, and ASQ:SE.
Reference standard assessment tools: 1) Baseline assessment of developmental quotient will be made using the Mullen Scales of Early Learning [40], which is a standardised assessment of cognition from birth to 68 months. The scales show strong and continuous validity over time and across cultures and are widely used with pre-school children; 2) The Autism Diagnostic Observation Schedule 2nd edition–Toddler Module (ADOS) [41] is a semi-structured, standardised observational assessment of the child’s communication, social interaction, and play. This instrument has been designed to assist in the diagnosis of ASD, with a diagnostic algorithm generated that is consistent with the primary diagnostic classification systems. The instrument has excellent inter-rater agreement in diagnostic classification, good test-retest reliability and internal consistency. To ensure the reliability of the ADOS assessment, we will video all assessments and ten percent of the video samples will be randomly selected and reviewed by an independent assessor to assess inter-rater reliability. Once all the assessments are completed, this will be reviewed by a panel with clinical expertise in paediatrics, psychiatry, child and family health nursing and speech pathology as appropriate, to determine a clinical decision about the outcome of the assessments. Families of children identified to have features consistent with a developmental disorder will also be provided with information about the need for further clinical assessment. Service mapping and a local service directory of resources will be provided along with referrals to the local multidisciplinary assessment clinics covering the catchment areas of the two teaching hospitals.
Once these assessments are conducted we will calculate the test characteristics (sensitivity, specificity, positive and negative likelihood ratios) of 1) the PEDS alone, 2) the current NSW surveillance program (PEDS with second level ASQ); and 3) the PEDS plus M-CHAT by comparing the performance of these tools, alone or in combination, in correctly identifying children with a developmental disorder.
Power calculation
A sample size of 500 children is large compared to the majority of previous studies exploring test performance of developmental surveillance tools and combinations of tools. The PEDS has been reported to have a sensitivity and specificity of around 70 to 80% [20]. Assuming the prevalence of developmental problems in the population is 10%, a total sample size of 449 subjects will be required to achieve a confidence interval width of 12.5% (that is, precision) around a minimum sensitivity of 76% [42]. A smaller sample size will be required for a similar precision with a minimum specificity of 75% (n = 52). Allowing for a dropout rate of approximately 10%, a total of 500 subjects will provide sufficient experimental power.
Data quality checks
A large proportion of the NBQs will be completed by the parents at the postnatal wards at recruitment and the remainder, along with the PDSQs will be completed over the telephone. To ensure that the questions are answered in a format that is suitable for analysis and the researchers’ questionnaire completion is consistent, each questionnaire will be checked prior to data entry. Any missing or ambiguous answers will be re-asked at a subsequent telephone call or when researchers see the parents at the developmental assessments.
Data linkage
It is possible that we will miss some mothers in recruitment for reasons such as: delivery during weekends or public holidays, early discharge following delivery, and refusal to participate in the study. Thus it is important to ascertain the representativeness of the sample that will be recruited. This will be done through data linkage of our cohort with electronic medical records of all the mothers who delivered children during the recruitment period at the two hospitals. We will compare key socio-demographic factors such as age, education level, indigenous background, postcode of residence, country of birth, and languages spoken between the mothers recruited to the project and those who are not recruited.
Similarly, for the prospective cohort component, we will use data linkage to compare the rates of those who attend the 6-, 12-, and 18-month developmental checks from the study participant group with those who are not participants in the study. It is possible that women who agree to participate are more aware of and engaged with health services (including the surveillance program) as compared to those who are not participating in the study. Hence, relationships between various sociodemographic factors and attendance at a 6-, 12-, and 18-month health and/or developmental check with a healthcare professional will also be examined using data linkage with community nurse records. Similarly we will compare key sociodemographic factors for those who drop out of the study as compared to those who continue in the study until their infants reach 18 months of age.
Ethics
Ethical approval to perform the study has been obtained from the University of New South Wales Human Research Ethics Committee.
Promotion of the project
During the recruitment period, clinical and administrative staff will be informed about the study through clinical and research meetings as well as by displaying posters with information about the study on the wards and waiting rooms. A website has also been developed.
Management of project/governance
A Project Management Committee has been established with representatives from each of the partner organisations and quarterly meetings of this committee will provide the forum for strategic oversight, reporting and feedback. Further, a Research Implementation Committee has been formed to oversee the day-to-day operational aspects of the study. Monthly meetings will be held to discuss issues on an ongoing basis for each of the core components of the study, that is, qualitative focus groups, longitudinal follow-up, and 18-month diagnostic test accuracy components. Other stakeholders and national and international experts will be invited to participate in the policy translation phase.