- Research article
- Open Access
Development process and cognitive testing of CARATkids - Control of Allergic Rhinitis and Asthma Test for children
BMC Pediatrics volume 14, Article number: 34 (2014)
Allergic rhinitis and asthma (ARA) are chronic inflammatory diseases of the airways that often coexist in children. The only tool to assess the ARA control, the Control of Allergic Rhinitis and Asthma Test (CARAT) is to be used by adults. We aimed to develop the Pediatric version of Control of Allergic Rhinitis and Asthma Test (CARATkids) and to test its comprehensibility in children with 4 to 12 years of age.
The questionnaire development included a literature review of pediatric questionnaires on asthma and/or rhinitis control and two consensus meetings of a multidisciplinary group. Cognitive testing was carried out in a cross-sectional qualitative study using cognitive interviews.
Four questionnaires to assess asthma and none to assess rhinitis control in children were identified. The multidisciplinary group produced a questionnaire version for children with 17 questions with illustrations and dichotomous (yes/no) response format. The version for caregivers had 4-points and dichotomous scales. Twenty-nine children, 4 to 12 years old, and their caregivers were interviewed. Only children over 6 years old could adequately answer the questionnaire. A few words/expressions were not fully understood by children of 6 to 8 years old. The drawings illustrating the questions were considered helpful by children and caregivers. Caregivers considered the questionnaire complete and clear and preferred dichotomous over the 4-points scales. The proportion of agreement between children and their caregivers was 61%. The words/expressions that were difficult to understand were amended.
CARATkids, the first questionnaire to assess a child’s asthma and rhinitis control was developed and its content validity was assured. Cognitive testing showed that CARATKids is well-understood by children 6 to 12 years old. The questionnaire’s measurement properties can now be assessed in a validation study.
Allergic rhinitis and asthma (ARA) are chronic inflammatory diseases of the airways that often coexist. The concept of “one airway one disease” was highlighted in the Allergic Rhinitis and its Impact on Asthma (ARIA) guidelines and the importance of an appropriate strategy combining safe and effective management of both diseases, targeting optimal control, in adults and children, was emphasized .
Questionnaires can be used as objective tools to evaluate disease control. For adults, there are several questionnaires that were developed to assess asthma control [2–4]. In what concerns allergic rhinitis the concept of control is still under definition ; nevertheless, some questionnaires have been proposed (Rhinitis Control Assessment Test (RCAT)  and Allergic Rhinitis Control Test ). For children, although the Practical Allergy (PRACTALL) consensus report  emphasized the use of these tools to monitor asthma control, few questionnaires are available [9–12]. And none to assess rhinitis control.
Based on ARIA recommendations, a single questionnaire should evaluate the control of asthma and allergic rhinitis [1, 13]. To account for this need, the Control of Allergic Rhinitis and Asthma Test (CARAT) was developed and validated [14–16] and it has been proposed as the first tool implementing ARIA guidelines in clinical practice . However, it was only validated for adult patients  and such combined tool was still missing for children. To address this problem we aimed to develop the CARATkids - Control of Allergic Rhinitis and Asthma Test for children – a questionnaire to concurrently assess control of allergic rhinitis and asthma in children under 12 years old, with a medical diagnosis of ARA. This article reports the process of development of the CARATkids and the results of the cognitive testing that lead to the preliminary version of the questionnaire.
The development of the test version of CARATkids was performed in three steps (Figure 1), including a literature review and two consensus meetings.
A bibliographic search was conducted in MEDLINE using as search terms: child/children, asthma, rhinitis, questionnaire and/or control. Articles addressing questionnaire development and/or validation were selected and retrieved. References of the selected articles were searched for additional data sources. The data extracted included: age group and disease(s) of the target population, item generation methods, intended respondent, proportion of agreement between child and caregiver when applicable, number of questions, time frame, answering options (type of scales used), the presence of drawings and items assessed.
First consensus meeting
A working group reviewed the existing data and proposed a pool of questions on control of ARA in children. This working group of 26 Portuguese experts in ARA, including Pediatricians, General Practitioners, Pulmonologists and Allergologists, participated in a consensus meeting, held in January 2010, where data on existing pediatric questionnaires and the initial (CARAT17 ) and reduced versions of CARAT (CARAT10 [15, 16]) for adults were discussed. In this meeting, 3 groups independently analyzed the existing questionnaires and proposed questions to be included in CARATkids. After pooling the proposed questions, it was unanimously decided to use CARAT17 as the basis for the first draft version of CARATkids as it covered all the questions proposed (Figure 1).
The expert panel considered important that, at least during the development of the questionnaire, both child’s and their caregivers’ input were gathered using a similar set of questions. This would imply two versions of the questionnaire: one to be answered by the child and another by the caregiver.
Second consensus meeting
In January 2011 the first draft of the questionnaire was presented to the expert panel. The draft was discussed, suggestions were presented, changes were made and a test version of the questionnaire approved (Figure 1).
This development process was designed to assure the content validity of the CARATkids questionnaire, according to the recently published COSMIN (COnsensus-based Standards for the selection of health Measurement Instruments) checklist .
The cognitive testing was a cross-sectional, observational, qualitative study with face-to-face interview of a convenience sample of 29 children and their parents/caregivers. Children with 4 to 12 years old with medical diagnosis of allergic rhinitis and asthma (Figure 2) were enrolled between July and December 2011, during regular appointments at the Allergy outpatient clinic of the Pediatric Hospital Dona Estefânia, in Lisbon, Portugal; children and caregivers were eligible if they were native Portuguese speakers. The interviews were performed independently by a psychologist and a physician. Children and their caregivers were interviewed separately. The participants were asked to read each question and response aloud; when the children couldn’t read by him/herself, the questions were read aloud by the interviewers. The answers were then challenged to make sure that the questions had been correctly understood. The child’s version was also discussed with the caregivers.
This study was conducted according to the principles expressed in the Declaration of Helsinki. Verbal informed consent was obtained from the caretakers of all children participating in survey. These procedures were approved by the institutional Ethical Review Board of the Hospital CUF Descobertas, Lisbon, Portugal.
Four questionnaires on asthma control were retrieved (The Asthma Quiz for Kidz , Childhood Asthma Control Test (C-ACT), Asthma Therapy Assessment Questionnaire (ATAQ) for Children and Adolescents , Test for Respiratory and Asthma Control in Kids (TRACK) ). No questionnaires on control of rhinitis or concurrent assessment of ARA for children were found.
Two versions of the CARATkids questionnaire were developed (child’s and caregiver’s) with slight differences in wording and response options.
A 17-item questionnaire with dichotomous (Yes/No) answer format accompanied by an illustrative colored drawing was prepared. The dichotomous answer, similar to “The Asthma Quiz for Kidz” and ATAQ for children, was considered preferable for a child’s questionnaire as it is easy to understand and answer.
In the child’s version, the expression of a specific time-frame was included only in four questions. The expert panel considered the previous 2 weeks as the most adequate time-frame to be answered by children. In relation to CARAT17 for adults, questions’ and headings’ vocabulary was adapted and/or simplified to be more appropriate and easily understood by children (e.g.: “school” instead of work).
Regarding the colored drawings for each question, it was consensual to have only one version for both boys and girls; the image of a boy was selected. The drawings were discussed by the expert panel and improved until approval.
In the questionnaire for caregivers all questions had a time-frame of 2 weeks. The caregiver version of the questionnaire kept the format of CARAT17, namely a 4-points Likert scale in questions 1 to 15 and a dichotomous scale in questions 16 and 17. To make clear that this questionnaire related to the child, all the questions in the caregiver’s version included the expression “(…) your child (…)”.
The characteristics of the children included in the cognitive testing are summarized in Table 1.
Children with 4 to 5 years old were not able to read by themselves and could not fully understand the questionnaire. Children with 6 years or older were able to understand the questions; however, a few words, such as “rhinitis” or “symptom” were not known by children with 6 to 8 years. Moreover, the words/expressions “dyspnea” and “eye weep” were not understood by several children, irrespective of age, and were removed. A few words were replaced by synonyms that were more easily understood by children and the sentence “respiratory/allergic diseases” was abridged to “allergies”. All children older than 9 years considered the questionnaire very easy and clear. The children enjoyed the drawings and found them clear and illustrative of the concepts. None of the children or caregivers had questions or concerns about the drawings.
Caregivers agreed that the questionnaire was complete and no suggestion for additional questions were proposed. They considered that the 2 weeks period time was adequate. When considering the child’s version, caregivers reported it to be clear and adequate; they appreciated the dichotomic scale, considering it to be more appropriate for their children, compared to the 4-point Likert scale of the caregiver’s questionnaire. Most of the caregivers were in favor of the inclusion of time frame in the questions of the child’s version. Similarly, some children opposed to the absence of a time frame in the child’s version and older children specifically asked for how long should they think about; this fact may have caused some disagreement between the caregivers’ and child’s answers. Nevertheless, four (out of 6) of the 6 years old children didn’t understand the time-frame of 2 weeks.
The proportion of agreement between children and their caregivers was 61%; none of the child/caregiver pairs agreed in all the answers. The symptoms of nasal obstruction and throat itching were less reported by caregivers than by their children.
The test version for children, revised after the cognitive testing, is presented in Figure 3.
The process of development of the CARATkids questionnaire included literature review, two consensus meetings with a multidisciplinary expert panel and face-to-face interviews with children and their caregivers for cognitive testing. The cognitive testing showed that the test version of CARATkids was easily understood and answered by both caregivers and children with 6 years or older. Younger children (4 to 5 years) were not able to fully understand the questionnaire.
Most of the existing questionnaires evaluating asthma control in children were developed based on clinical guidelines (Table 2) and none assessed asthma and allergic rhinitis concurrently. Within available questionnaires, CARAT17 was the only one assessing the control of both asthma and allergic rhinitis and was based on a comprehensive item generation process  making it suitable to be the basis for development of CARATkids.
Moreover, the expert panel and the caregivers (in the cognitive testing) had no suggestion for other questions to be included or existing questions to exclude: this supports the relevance of the included items.
The importance of incorporating the child’s perspective when evaluating asthma control has been highlighted in several studies [9, 10]; however, in younger children (especially when they are unable to read by themselves or don’t know how to read properly), the parent/caregiver’s input may be essential . In fact, we found that children younger than 6 were unable to adequately understand the questionnaire, suggesting that, in this age group, ARA control may be best assessed by a tool to be answered by the caregiver alone or with the child’s help. In older children, the independent input of children and caregivers ensures a broader perspective of diseases’ control and overcomes the limitations associated with relying in one isolated report, either from the child or the parent. Supporting this decision, the cognitive testing of CARATkids showed 39% disagreement between the symptoms reported by the child and caregiver; this was already reported by other studies [10, 21]. The inclusion of nasal and throat symptoms (e.g.: nasal obstruction and throat itching) may have heightened this disagreement. These symptoms are frequently long-lasting but mild enough not to interfere majorly with the children’s daily activities; therefore, caregivers often don’t notice nasal or throat symptoms or regard them as “normal” or as part of a persistent cold and not as a sign of an allergic disorder. Nevertheless, the agreement between children and caregivers answers should be reassessed in future studies with the final version of the questionnaire.
The time frame used for assessment of control in existing pediatric questionnaires is highly variable and depends on the input (s) included (child vs. caregiver vs. child and caregiver). However it seems consensual that in child’s questions it is more appropriate and reliable to evaluate short time periods. Childhood-ACT , for example, uses “today” in questions to be answered for children because in a round of interviews with children aged 4–6 years, they presented difficulties recalling beyond 1 day. In the existing questionnaires, none of the questions to be answered by children evaluates more than 4 weeks (Table 2). In the development of the CARATkids questionnaire it was decided to eliminate the time frame from most of the questions based on the knowledge that children, especially the younger/pre-school, may be unable to remember or even understand the concept of one week or one month. Only those questions regarding the use of rescue medication, unscheduled medical visit and hospital admission (more “relevant” events) reported specifically to the last 2 weeks. This option was very much discussed during consensus meetings and by some children and caregivers in the cognitive interviews and should be further studied.
The CARATkids questionnaire was designed with a dichotomous-scale answer format, similar to The Asthma Quiz for Kidz  and ATAQ for children . This answer format was considered adequate by children and parents.
The inclusion of drawings illustrating each question was also appreciated by the participants. This kind of visual support was already used in The Asthma Quiz for Kidz ; the C-ATC  also includes drawings, but only to illustrate the answer options. It was consensually decided to use the image of a boy and neither children nor caregivers participating in the cognitive testing of CARATkids reported questions related to the use of a boy’s image; previous research  has shown that the boy’s face is preferred by the majority of children (vs. generic or girl’s face).
The cognitive test assured that the questionnaire was adequately understood by both children and caregivers and allowed to improve words and expressions that were not sufficiently clear. This was a fundamental step in the development of this questionnaire, providing in-depth data, that was a major contribute to build a clear and appealing questionnaire. Moreover, it allowed the assessment of its feasibility in the target age groups. The comments of the caregivers supported the idea that CARAT is an useful and adequate questionnaire to assess asthma and rhinitis control. This process may seem limited by the small number of participating children, especially in the group with 4 to 5 years old. However, as a qualitative methodology, cognitive interviews should be stopped when no more new information is being obtained. For children with 4 to 5 years old with a few interviews was clear that the type of questionnaire we were developing could not be applied. Therefore we stop to interview children from that age group.
As far as we know, CARATkids is the first questionnaire aiming to assess control of asthma and allergic rhinitis concurrently in children. It is being developed following the COSMIN check-list  that evaluates the methodological quality of studies on measurement properties. Although this is not specific for childhood questionnaires, we believe that the COSMIN initiative  can help to improve the development and selection of health measurement tools. Additional knowledge, specifically regarding the development of pediatric questionnaires, should be sought.
This article describes the aspects assuring the content validity  of the CARATkids questionnaire. Additional studies are needed to evaluate its measurement properties. A clinical prospective validation study is underway.
CARATkids, the first questionnaire to assess a child’s asthma and rhinitis control was developed and its content validity was assured. Cognitive testing was important to guarantee that CARATKids is well-understood by children 6 to 12 years old. The questionnaire’s measurement properties can now be assessed in a prospective validation study.
Allergic rhinitis and asthma
Allergic rhinitis and its impact on asthma
Asthma therapy assessment questionnaire
Childhood asthma control Test
Control of allergic rhinitis and asthma test
Pediatric version of the control of allergic rhinitis and asthma Test
COnsensus-based standards for the selection of health measurement instruments
Global initiative for asthma
National asthma education and prevention program
Rhinitis control assessment Test
Test for respiratory and asthma control in kids.
Bousquet J, Van Cauwenberge P, Khaltaev N: Allergic rhinitis and its impact on asthma. J Allergy Clin Immunol. 2001, 108 (5 Suppl): S147-S334.
Nathan RA, Sorkness CA, Kosinski M, Schatz M, Li JT, Marcus P, Murray JJ, Pendergraft TB: Development of the asthma control test: A survey for assessing asthma control. J Allergy Clin Immunol. 2004, 113 (1): 59-65. 10.1016/j.jaci.2003.09.008.
Juniper EF, O'Byrne PM, Guyatt GH, Ferrie PJ, King DR: Development and validation of a questionnaire to measure asthma control. Eur Respir J. 1999, 14 (4): 902-907. 10.1034/j.1399-3003.1999.14d29.x.
Boulet LP, Boulet V, Milot J: How should we quantify asthma control?. A proposal. Chest. 2002, 122 (6): 2217-2223. 10.1378/chest.122.6.2217.
Demoly P, Calderon MA, Casale T, Scadding G, Annesi-Maesano I, Braun JJ, Delaisi B, Haddad T, Malard O, Trebuchon F, et al: Assessment of disease control in allergic rhinitis. Clin Transl Allergy. 2013, 3 (1): 7-10.1186/2045-7022-3-7.
Schatz M, Meltzer EO, Nathan R, Derebery MJ, Mintz M, Stanford RH, Dalal AA, Silvey MJ, Kosinski M: Psychometric validation of the rhinitis control assessment test: a brief patient-completed instrument for evaluating rhinitis symptom control. Ann Allergy Asthma Immunol. 2010, 104 (2): 118-124. 10.1016/j.anai.2009.11.063.
Demoly P, Jankowski R, Chassany O, Bessah Y, Allaert FA: Validation of a self-questionnaire for assessing the control of allergic rhinitis. Clin Exp Allergy. 2011, 41 (6): 860-868. 10.1111/j.1365-2222.2011.03734.x.
Bacharier LB, Boner A, Carlsen KH, Eigenmann PA, Frischer T, Gotz M, Helms PJ, Hunt J, Liu A, Papadopoulos N, et al: Diagnosis and treatment of asthma in childhood: a PRACTALL consensus report. Allergy. 2008, 63 (1): 5-34.
Ducharme FM, Davis GM, Noya F, Rich H, Ernst P: The Asthma Quiz for Kidz: a validated tool to appreciate the level of asthma control in children. Can Respir J. 2004, 11 (8): 541-546.
Liu AH, Zeiger R, Sorkness C, Mahr T, Ostrom N, Burgess S, Rosenzweig JC, Manjunath R: Development and cross-sectional validation of the Childhood Asthma Control Test. J Allergy Clin Immunol. 2007, 119 (4): 817-825. 10.1016/j.jaci.2006.12.662.
Skinner EA, Diette GB, Algatt-Bergstrom PJ, Nguyen TT, Clark RD, Markson LE, Wu AW: The Asthma Therapy Assessment Questionnaire (ATAQ) for children and adolescents. Dis Manag. 2004, 7 (4): 305-313. 10.1089/dis.2004.7.305.
Murphy KR, Zeiger RS, Kosinski M, Chipps B, Mellon M, Schatz M, Lampl K, Hanlon JT, Ramachandran S: Test for respiratory and asthma control in kids (TRACK): a caregiver-completed questionnaire for preschool-aged children. J Allergy Clin Immunol. 2009, 123 (4): 833-839. 10.1016/j.jaci.2009.01.058. e839
Bousquet J, Schünemann HJ, Zuberbier T, Bachert C, Baena-Cagnani CE, Bousquet PJ, Brozek J, Canonica GW, Casale TB, Demoly P, et al: Development and implementation of guidelines in allergic rhinitis – an ARIA-GA2LEN paper. Allergy. 2010, 65 (10): 1212-1221. 10.1111/j.1398-9995.2010.02439.x.
Nogueira-Silva L, Martins SV, Cruz-Correia R, Azevedo LF, Morais-Almeida M, Bugalho-Almeida A, Vaz M, Costa-Pereira A, Fonseca JA: Control of allergic rhinitis and asthma test - a formal approach to the development of a measuring tool. Respir Res. 2009, 10: 52-10.1186/1465-9921-10-52.
Fonseca JA, Nogueira-Silva L, Morais-Almeida M, Azevedo L, Sa-Sousa A, Branco-Ferreira M, Fernandes L, Bousquet J: Validation of a questionnaire (CARAT10) to assess rhinitis and asthma in patients with asthma. Allergy. 2010, 65 (8): 1042-1048. 10.1111/j.1398-9995.2009.02310.x.
Fonseca JA, Nogueira-Silva L, Morais-Almeida M, Sa-Sousa A, Azevedo LF, Ferreira J, Branco-Ferreira M, Rodrigues-Alves R, Bugalho-Almeida A, Bousquet J: Control of Allergic Rhinitis and Asthma Test (CARAT) can be used to assess individual patients over time. Clin Transl Allergy. 2012, 2 (1): 16-10.1186/2045-7022-2-16.
Azevedo P, Correia De Sousa J, Bousquet J, Bugalho-Almeida A, Del Giacco SR, Demoly P, Haahtela T, Jacinto T, Garcia-Larsen V, Van Der Molen T, et al: Control of Allergic Rhinitis and Asthma Test (CARAT): dissemination and applications in primary care. Prim Care Respir J. 2013, 22 (1): 112-116. 10.4104/pcrj.2013.00012.
Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HC: The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010, 19 (4): 539-549. 10.1007/s11136-010-9606-8.
National Asthma Education and Prevention Program: Expert Panel Report 3 (EPR-3): Guidelines for the Diagnosis and Management of Asthma-Summary Report 2007. J Allergy Clin Immunol. 2007, 120 (5 Suppl): S94-S138.
Boulet LP, Becker A, Berube D, Beveridge R, Ernst P: Canadian Asthma Consensus Report, 1999. Canadian Asthma Consensus Group. Can Med Assoc J. 1999, 16 (11 Suppl): S1-S61.
Lara M, Duan N, Sherbourne C, Lewis MA, Landon C, Halfon N, Brook RH: Differences between child and parent reports of symptoms among latino children with asthma. Pediatrics. 1998, 102 (6): e68-10.1542/peds.102.6.e68.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2431/14/34/prepub
The authors would like to thank the children and caregivers and the physicians that participated in the study and the Sociedade Portuguesa de Alerglogia e Imunologia Clínica and Associação Portuguesa de Asmáticos e Alérgicos for the help and support the CARAT project. The consensus meetings had logistic and unrestricted financial support from MSD Portugal.
The authors declare that they have no competing interests.
LMB participated in the study conception, data collection and interpretation and wrote the manuscript draft, JAF participated in the study conception, data interpretation and in the writing of the manuscript, AMP and DL participated in data interpretation and in the writing of the manuscript draft, VRP participated in the data collection, MMA participated in the study conception and provided critical review during the project. All authors have reviewed and approved the final manuscript.
About this article
Cite this article
Borrego, L.M., Fonseca, J.A., Pereira, A.M. et al. Development process and cognitive testing of CARATkids - Control of Allergic Rhinitis and Asthma Test for children. BMC Pediatr 14, 34 (2014). https://doi.org/10.1186/1471-2431-14-34
- Cognitive testing