- Research article
- Open Access
- Open Peer Review
Determinants of response to a parent questionnaire about development and behaviour in 3 year olds: European multicentre study of congenital toxoplasmosis
© Salt et al; licensee BioMed Central Ltd. 2005
- Received: 27 October 2004
- Accepted: 05 July 2005
- Published: 05 July 2005
We aimed to determine how response to a parent-completed postal questionnaire measuring development, behaviour, impairment, and parental concerns and anxiety, varies in different European centres.
Prospective cohort study of 3 year old children, with and without congenital toxoplasmosis, who were identified by prenatal or neonatal screening for toxoplasmosis in 11 centres in 7 countries. Parents were mailed a questionnaire that comprised all or part of existing validated tools. We determined the effect of characteristics of the centre and child on response, age at questionnaire completion, and response to child drawing tasks.
The questionnaire took 21 minutes to complete on average. 67% (714/1058) of parents responded. Few parents (60/1058) refused to participate. The strongest determinants of response were the score for organisational attributes of the study centre (such as direct involvement in follow up and access to an address register), and infection with congenital toxoplasmosis. Age at completion was associated with study centre, presence of neurological abnormalities in early infancy, and duration of prenatal treatment. Completion rates for individual questions exceeded 92% except for child completed drawings of a man (70%), which were completed more by girls, older children, and in certain centres.
Differences in response across European centres were predominantly related to the organisation of follow up and access to correct addresses. The questionnaire was acceptable in all six countries and offers a low cost tool for assessing development, behaviour, and parental concerns and anxiety, in multinational studies.
- Infected Child
- Neonatal Screening
- Organizational Attribute
- Parental Concern
- Parental Anxiety
Measurement of children's development, behaviour, and impairment is essential in studies that seek to determine the impact of early life events on functional abilities. Because professional administered standardised assessments are extremely resource intensive, parent-completed questionnaires are used increasingly, particularly in large studies of populations at low risk of impairment [1–4]. Uncertainties about the validity of parent-reported outcomes have been addressed by several studies showing that, compared with professional assessments, parents correctly report moderate to severe cognitive or speech and language impairment, behavioral problems, and disability [3, 5–12]. Much less is known about the reliability and acceptability of parent-completed questionnaires in different countries, languages and cultures, except for tools measuring behaviour or quality of life [13–17]. Such information is particularly relevant for multinational studies.
This report is based on a prospective multicenter cohort study, The European Multicentre Study on Congenital Toxoplasmosis (EMSCOT), which was initiated to determine the effects of congenital toxoplasmosis and prenatal treatment on development, behaviour, specific impairments, and parental anxiety. We were constrained by the need for the assessment tool to be low cost, require minimal input by local investigators, cover all domains of development, and avoid measurement of vocabulary or other language-specific attributes. In addition, we wanted a tool that maximised response, minimised bias among responders, measured the same entity, and was similarly acceptable, in all six countries studied. The aim of the tool was to detect moderate to severe abnormality in the outcomes measured.
In order to assess the potential for bias when using the postal questionnaire, we examined the influence of organizational factors within centers and characteristics of the individual child, on three outcomes: response to the questionnaire, the age at response, and, among responders, completion of the drawing tasks by children. The aim was to determine the applicability and acceptability of this low cost tool in different European settings.
Summary of clinical management and follow up protocols, and response rates for each center
Total sent questionnaires
Prenatal re-testing interval (months)1
Duration post-natal treatment (month)
Postnatal follow up
All children born to toxoplasma infected women had paediatric, ophthalmic and cranial ultrasound examinations in early infancy, and infected children were assessed annually . The exception to this rule was the group of uninfected children in Poznan who were not offered specialist clinical follow up. At 36 months of age, a questionnaire was mailed to parents together with a stamped addressed reply envelope, an information sheet, and crayons for the child. Two reminders were mailed to non-responders at 2 monthly intervals.
After the study, we sent a questionnaire to each centre to measure organisational attributes, such as whether local study investigators were directly involved in provided clinical follow up for the child, whether they had regular contact with the child's own paediatrician, access to a central address register for tracing families, and contact with families to encourage response. These factors were summed to generate a total unweighted score (see Table 1).
Source for questions measuring development, behavior, and parental concerns and anxiety
Question number in questionnaire*
Development and behavior
Griffiths Mental Development Scales , Denver Developmental screening test 
General Language Screen**, parent completed questionnaire for 3 year olds . Validated against four standardised speech and language tests administered by an assessor.
Cognitive ability (non-verbal)
PARCA3** (Parent Report of Children's Abilities)  validated for 3 year olds against the MacCarthy Scale
'Strengths and difficulties questionnaire' (SDQ), validated in 3 to 16 year olds against clinician assessment of behavior disorder [10,12]. Entire questionnaire, published translations, and scoring algorithm used.
Parental concerns, specialist referral, and parental anxiety
a) Learning, behavior, development
b) Speech and language
Adapted from PEDS** (Parent Evaluation of Developmental Status). Predicts risk for developmental and behavioral problems and the need for clinical assessment.[26,27]
Impact of behavior on family
SDQ questionnaire [10,12,28]
Adaptation of rating scales for measuring anxiety during pregnancy and postpartum  in relation to antenatal screening (numbered six point horizontal scale with verbal anchors at extremes).
Child completed questions
Cognitive and fine motor skills
Copying a line, circle, and cross.
Draw a man
The child's ability to copy a circle, line and cross was assessed using scoring and normative data available from the Beery Buktenica Developmental Test of Visual Motor Integration . The 'draw a man' was scored using a standardised system and normative data from the Goodenough Draw a Man test , using raw scores.
Education level achieved
Educational level achieved based on standard categories defined by the Organisation for Economic Co-operation and Development (OECD) for Europe.
Questionnaire development and piloting
Development of the questionnaire involved collaborating pediatricians, obstetricians, parasitologists and psychologists in different countries to ensure that questions would be widely understood and acceptable. Questionnaires were translated into the six languages in the study, back translated to English by someone unaware of the original English version, and compared to the original version to ensure meaning was retained. The questionnaire was piloted in the six countries in general pediatric outpatient clinics, high risk (preterm) follow-up clinics, and day care centers, and parents were asked about difficult or offensive questions, the length of the questionnaire, and how long it took to complete. Research ethics approval was obtained for all participating centres
Development of scores
To summarize the responses relating to development and parental anxiety, unweighted scores were derived without knowledge of infection or treatment status. Spearman correlation coefficients were used to identify redundancies among items, and the final scale was based on items with relatively low inter-item correlations. If less than 50% of answers were missing for each outcome, the total score was prorated. For behaviour, and the children's drawings, we used the published scoring systems (see Table 3). All scores were coded so that a high score was abnormal. One third of the children's drawings were scored by a second assessor and discrepancies reviewed.
Analysis of response
We developed multivariate models to identify factors associated with each of the three outcomes: a) whether the questionnaire was completed and returned; b) the child's age at questionnaire completion; and c) whether the child completed the 'draw a man' task. Age at response was analysed as a surrogate marker for 'response or not' that might be susceptible to family as well as center factors. The child-completed task was included to assess its acceptability and the potential bias involved in such assessments that require additional effort from the family.
Initially, we examined the heterogeneity of effects within French centres, and found significant differences in response across centres. Thus, we decided to use a hierarchical generalized linear model for dichotomous outcomes (response to the survey, and response to 'Draw a Man') to account for heterogeneity among centers within France, along with other centres in the model. A generalized estimating equation (SAS Version 9.1 PROC GENMOD with the ASSESS options to assess fit of the model) with centers nested within country was derived to determine characteristics associated with response to questionnaire, and completion of the 'draw a man' task. Goodness of fit was assessed using the Pearson Chi-square result divided by its degrees of freedom. Values close to 1 indicated lack of overdispersion of the model . To predict child's age at survey completion, we used multiple linear regression. Lyon, the largest centre, was used as the reference category.
Characteristics associated with response to questionnaire (N = 1058 total)
Number responding (%)
Odds ratio for response4 (95% confidence interval)
Final model: Adjusted odds ratio5 (95% CI)
0.37 (0.24, 0.57)
0.11 (0.05, 0.27)
1.16 (0.65, 2.06)
1.12 (0.53, 2.38)
0.56 (0.31, 0.99)
0.67 (0.21, 2.1)
0.97 (0.61, 1.54)
0.37 (0.13, 1.05)
6.22 (1.85, 20.84)
1.09 (0.68, 1.73)
Total score for organisational attributes 3
1.36 (1.27, 1.45)
1.15 (1.09, 1.23)
Infection status 1
2.95 (2.01, 4.31)
4.96 (3.58, 6.88)
Maternal age 1,2,3 (mean years, 95% CI)
582 (67%) 28.4 (27.6, 29.2)
1.03 (0.99, 1.07)
1.02 (0.99, 1.05)
Parity 1,2,3 (mean, 95% CI)
503 (67%) 0.9 (0.8, 1.0)
1.05 (0.90, 1.23)
Gestational age at birth 1,2,3 (mean weeks, 95% CI)
530 (68%) 39.0 (38.8, 39.1)
1.00 (0.92, 1.09)
Child's gender 1,2
0.94 (0.71, 1.25)
Prenatal treatment 1,2
1.85 (0.98, 3.49)
Neurological abnormality and/or intracranial lesions 1,2
2.69 (0.56, 13.00)
As the total score for organizational attributes (given in Table 1) was a proxy for centre, we repeated all analyses, initially adjusting for centre, and then adjusting for the total unweighted score for organizational attributes. Potential covariates were added to a model with congenital infection status and center to determine the magnitude of association with outcome. Variables with associations that resulted in p-values less than .20 were included in the initial multivariate model. A monitored backwards stepwise approach was conducted, and models were assessed for convergence. The final model included only variables (or categories of variables) significant at p < .05. Bivariate associations were assessed using a Chi-square or Exact test for categorical characteristics and Wilcoxon Rank Sum tests for ordinal or non-normally distributed characteristics. The best fitting, and most parsimonious model was included in the results presented. Odds ratios or estimated means are presented along with 95% confidence intervals.
Piloting of questionnaire
115 parents completed pilot questionnaires (70 healthy, 31 seen in pediatric clinics for clinical problems, 14 not specified), in France (32), Italy (40), Sweden (9), Denmark (10), and Poland (24). On average, parents took 21 minutes to complete the questionnaire (range 6 to 40 minutes, SD 7.88). Most parents (88%) thought that the questionnaire was the right length, 5% reported it was short, and 7% too long. Sixteen (14%) had difficulty understanding, or objected to one or more questions, of which the most frequent were: 'maternal age when last in full time education', questions about behavior (three parents felt the questions were not suitable for the age group), and one cognitive question about puzzles which was removed.
Survey of organizational attributes and reasons for non-response
Summary of response rates for each center
Reasons for non-response (% non-responders)
Response rate (%)
Address not known
Refused to participate
Analysis of determinants of response to questionnaire
As shown in Table 4, there were statistically significant differences between centres in the proportion of parents responding to the questionnaire. Response was more common in Naples, and less common in Paris, Grenoble and Toulouse, than in Lyon. A more parsimonious model involved replacement of the centre variable with the centre score for organisational attributes which was significantly associated with increased response. The only other significant factor in this model was congenital infection status. (see Table 4).
Determinants of age at response
On average the questionnaire was completed at 39.7 months of age (95% CI: 39.5, 40.0; range 35.4 to 63.9 months). In the multivariable analysis, factors significantly associated with older age at completion were study centre (delayed in Paris, Vienna and Naples), duration of prenatal treatment and detection of a neurological abnormality and/ or intracranial lesions in the first 4 months of life (mean difference in months at response was 1.67; with standard error = 0.72; R2 = 0.16.).
Determinants of child's response to 'draw a man'
94% of children copied drawings of a line, circle, and cross, but only 70% of children responded to the request to 'draw a man.' In multivariable analyses, completion of 'draw a man' was more common in girls (OR 0.62; 95% CI 0.44, 0.88), age at completion of the questionnaire (Odds ratio 1.13 per additional month of age; 95% CI 1.06–1.21), and centre (children in Poznan were more likely to respond than in Lyon; odds ratio 2.53, 95% CI 1.36, 4.68). The goodness of fit statistic was 1.0092.
Parent completed questions
Most questions (>99%) were completed. Questions with the lowest rate of completion were on hearing loss (94%), vision (92%), and age when mother was last in full time education (92%).
The score for organizational attributes varied between study centers and was one of the main determinants of response to the questionnaire. Centers where study clinicians were directly involved in patient follow up, had access to a central address register, and directly contacted parents to encourage return of the questionnaire, had the highest response rates. There was no evidence that organizational attributes were associated with age at response, nor with whether the child drew a man. Congenital infection status was strongly associated with response to the survey, but only weakly associated with age at response, and was not significantly associated with whether the child completed the 'draw a man' task.
The response rate to this parent report survey suggests that a parent-completed postal questionnaire on development, behavior, and parental concerns and anxiety is acceptable across the six European countries studied. The high response rate in this study was achieved by clinicians without dedicated research coordinators in the local centres, although there were dedicated staff centrally. In some centres clinicians were laboratory-based and not directly involved in follow up of the child. The results of this study should therefore be widely applicable.
Our findings concur with those of a systematic review of methods for increasing response rates to postal questionnaires ; response was higher among parents for whom the study was of most interest (those with infected children), and response was improved by follow-up contact. Other important elements of survey design highlighted in the review by Edwards et al included keeping the questionnaire short, and mailing a second copy.
Non-response can introduce bias if non-responders differ from responders with respect to prognostic characteristics. In our analyses, we found no evidence that non-responders differed in terms of maternal age, parity, or prenatal treatment or with respect to prognostic factors associated with poor developmental outcome such as gestational age at birth or abnormal clinical manifestations in early infancy. Difficulties tracing the correct address probably favored inclusion of infected children in our study, but we found no evidence for a bias in favor or against inclusion of more severely affected children.
The age at response was largely determined by the centre, and was not significantly associated with the organizational attributes. Although questionnaires were intended to be mailed as soon as they arrived at the local centre, actual practice may have varied. The centre effect may therefore be explained by unmeasured centre characteristics such as availability of staff to mail questionnaires. The weak association between duration of prenatal treatment and increased age at response may be a chance finding. However, return of questionnaires was delayed from children with an intracranial lesion or neurological abnormality. This may reflect difficulties contacting such families. A similar finding was reported in a cohort study of children born preterm; families with severely neurologically affected children were most difficult to contact (a higher proportion of families of children with severe disability repeatedly failed to attend appointments, moved frequently or were adopted or fostered) .
Although completion of individual questions was high, certain questions fared less well, most notably when the child was asked to 'draw a man'. Low response may be because the task is quite difficult for three year olds, the lower age limit for this test, as completion did improve with age. As child-completed tasks can provide additional objective information about development, inclusion of more age-appropriate tasks may be valuable [6, 7]. Failure of the 6% to 8% of parents to answer the questions on vision and hearing may reflect parental uncertainty or lack of confidence in reporting medical information whereas functional information was well reported. The question on 'maternal age when last in full time education' received most comments during the pilot phase, and was less popular than a related question about highest level of education achieved, which has been used most widely in Europe .
One of the main limitations of the study is that reasons for non-response were poorly documented. Consequently, refusals to participate may have been underestimated, and acceptability may have been overestimated. A further limitation is that study centre clinicians were surveyed about the organizational attributes of their centre after the study, when they were aware of how response rates differed among centers. This may have favored overestimation of the importance of organizational attributes for response.
The parent completed questionnaire was acceptable in 11 centres in six European countries. Differences in response appeared to be related to organisation of follow up, and access to correct addresses. The questionnaire offers a low cost tool for assessing development, behaviour, and parental concerns and anxiety, in multinational studies.
We are grateful to Robert Plomin, Frances Glascoe, and Carol Stott forallowing us to use adapted versions or items from their parentquestionnaires, to Robert Goodman for use of his questionanaire andtranslations, and to Bruno Senta-Loys (Lyon) for contributing ideas to thedesign of the questionnaire. The research was part of the European multicentre study on congenital toxoplasmosis, funded by the European Commission (BIOMED II No. BMH4-CT98-3927 and QLG5-CT-2000-00846). Additional support has been provided by the National Eye Institute, grant code R03 EY015287-01
- Johnson S, Marlow N, Wolke D, Davidson L, Marston L, O'Hare A, Peacock J, Schulte J: Validation of a parent report measure of cognitive development in very preterm infants. Dev Med Child Neurol. 2004, 46: 389-397. 10.1017/S0012162204000635.PubMedGoogle Scholar
- Doig KB, Macias MM, Saylor CF, Craver JR, Ingram PE: The Child Development Inventory: A developmental outcome measure for follow-up of the high-risk infant. J Pediatr. 1999, 135: 358-362.View ArticlePubMedGoogle Scholar
- Eisert DC, Spector S, Shankaran S, Faigenbaum D, Szego E: Mothers' reports of their low birth weight infants' subsequent development on the Minnesota Child Development Inventory. J Pediatr Psychol. 1980, 5: 353-364.View ArticlePubMedGoogle Scholar
- Petrill SA, Saudino K, Cherny SS, Emde RN, Fulker DW, Hewitt JK, Plomin R: Exploring the genetic and environmental etiology of high general cognitive ability in fourteen to thirty-six-month old twins. Child Dev. 1998, 69: 68-74.View ArticlePubMedGoogle Scholar
- Heiser A, Curcin O, Luhr C, Grimmer I, Metze B, Obladen M: Parental and professional agreement in developmental assessment of very-low-birthweight and term infants. Dev Med Child Neurol. 2000, 42: 21-24. 10.1017/S0012162200000050.View ArticlePubMedGoogle Scholar
- Oliver B, Dale DS, Saudino KJ, Petrill SA, Pike A, Plomin R: The validity of a parent-based assessment of cognitive abilities in three-year olds. Early Child Dev Care. 2002, 172: 337-348. 10.1080/03004430212713.View ArticleGoogle Scholar
- Saudino K, Dale P, Oliver B, Petrill S, Richardson V, Rutter M, Simonoff E, Stevenson J, Plomin R: The validity of parent-based assessment of the cognitive abilities of 2-year-olds. British Journal of Developmental Psychology. 1998, 16: 349-363.View ArticleGoogle Scholar
- Dale PS: The validity of a parent report measure of vocabulary and syntax at 24 months. J Speech Hear Res. 1991, 34: 565-571.View ArticlePubMedGoogle Scholar
- Dinnebeil L, Rule S: Congruence between parents' and professionals judgements about the development of young children with disabilities: A review of the literature. Topics in Early Childhood Special Education. 1994, 14: 1-25.View ArticleGoogle Scholar
- Glascoe FP, Sandler H: Value of parents' estimates of chldren's developmental ages. J Pediatr. 1995, 127: 831-835.View ArticlePubMedGoogle Scholar
- Gottfried AW, Guerin D, Spencer JE, Meyer C: Validity of Minnesota Child Development Inventory in screening young children's developmental status. J Pediatr Psychol. 1984, 9: 219-230.View ArticlePubMedGoogle Scholar
- Goodman R, Ford T, Simmons H, Gatward R, Meltzer H: Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Br J Psychiatry. 2000, 177: 534-539. 10.1192/bjp.177.6.534.View ArticlePubMedGoogle Scholar
- Smedje H, Broman JE, Hetta J, von Knorring AL: Psychometric properties of a Swedish version of the "Strengths and Difficulties Questionnaire". Eur Child Adolesc Psychiatry. 1999, 8: 63-70. 10.1007/s007870050086.View ArticlePubMedGoogle Scholar
- Klasen H, Woerner W, Wolke D, Meyer R, Overmeyer S, Kaschnitz W, Rothenberger A, Goodman R: Comparing the German versions of the Strengths and Difficulties Questionnaire (SDQ-Deu) and the Child Behavior Checklist. Eur Child Adolesc Psychiatry. 2000, 9: 271-276. 10.1007/s007870070030.View ArticlePubMedGoogle Scholar
- Pachter LM, Dworkin PH: Maternal expectations about normal child development in 4 cultural groups. Arch Pediatr Adolesc Med. 1997, 151: 1144-1150.View ArticlePubMedGoogle Scholar
- Anderson RT, Aaronson NK, Wilkin D: Critical review of the international assessments of health-related quality of life. Qual Life Res. 1993, 2: 369-395. 10.1007/BF00422215.View ArticlePubMedGoogle Scholar
- Perneger TV, Leplege A, Etter JF: Cross-cultural adaptation of a psychometric instrument: two methods compared. J Clin Epidemiol. 1999, 52: 1037-1046. 10.1016/S0895-4356(99)00088-8.View ArticlePubMedGoogle Scholar
- Gilbert R, Gras L: Effect of timing and type of treatment on the risk of mother to child transmission of Toxoplasma gondii. BJOG. 2003, 110: 112-120.View ArticlePubMedGoogle Scholar
- Zelterman D: Basic log-linear models and PROC Genmod procedure. Advanced log-linear models using SAS. Edited by: Zelterman D. 2002, Cary, North Carolina, SAS Institute Inc, 19-54.Google Scholar
- Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, Kwan I: Increasing response rates to postal questionnaires: systematic review. BMJ. 2002, 324: 1183-10.1136/bmj.324.7347.1183.View ArticlePubMedPubMed CentralGoogle Scholar
- Tin W, Fritz S, Wariyar U, Hey E: Outcome of very preterm birth: children reviewed with ease at 2 years differ from those followed up with difficulty. Arch Dis Child Fetal Neonatal Ed. 1998, 79: F83-F87.View ArticlePubMedPubMed CentralGoogle Scholar
- Mackenbach JP, Kunst AE: Measuring the magnitude of socio-economic inequalities in health: an overview of available measures illustrated with two examples from Europe. Soc Sci Med. 1997, 44: 757-771. 10.1016/S0277-9536(96)00073-1.View ArticlePubMedGoogle Scholar
- Griffiths R: The Griffiths Mental Development Scales from birth to eight years. 2004, Oxford, The Test Agency Ltd.Google Scholar
- Denver Developmental Screening Test. [http://www.denverii.com/]
- Stott CM, Merricks MJ, Bolton PF, Goodyer IM: Screening for speech and language disorders: the reliability, validity and accuracy of the General Language Screen. Int J Lang Commun Disord. 2002, 37: 133-151. 10.1080/13682820110116785.View ArticlePubMedGoogle Scholar
- Glascoe FP, Dworkin PH: The role of parents in the detection of developmental and behavioral problems. Pediatrics. 1995, 95: 829-836.PubMedGoogle Scholar
- Sonnander K: Parental developmental assessment of 18 month old children: reliability and predictive value. Dev Med Child Neurol. 1987, 29: 351-362.View ArticlePubMedGoogle Scholar
- Goodman R, Scott S: Comparing the Strengths and Difficulties Questionaire and the Child Behaviour Checklist: is small beautiful?. J Abnorm Child Psychol. 1999, 27: 17-24. 10.1023/A:1022658222914.View ArticlePubMedGoogle Scholar
- Marteau TM, Cook R, Kidd J, Michie S, Johnston M, Slack J, Shaw R: The psychological effects of false positive results in prenatal screening for fetal abnormality: a prospective study. Prenat Diagn. 1992, 12: 205-214.View ArticlePubMedGoogle Scholar
- Beery KE, Norman A: The Beery-Buktenica Developmental Test of Visual Motor Integration. 1997, New Jersey, Modern Curriculum PressGoogle Scholar
- Goodenough FL: Measurement of Intelligence by Drawings. 1926, New York, World Book CompanyGoogle Scholar
- Organisation for Economic Co-operation and Development (OECD).: Education in OECD Countries 1987-1988. A compendium of statistical information. 1990, ParisGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2431/5/21/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.