Reliability and validity of the FFQ and feeding index for 7-to 24-month-old children after congenital heart disease surgery
BMC Pediatrics volume 22, Article number: 348 (2022)
Congenital heart disease (CHD) is the most common congenital defect in neonates. Infants with CHD often have more nutritional difficulties, but currently, there is no unified Food Frequency Questionnaire (FFQ) for infants and young children aged 7–24 months in China. Therefore, we designed this study to assess the reliability and validity of the FFQ and feeding index for 7-to 24-month-old children after congenital heart disease surgery.
From July to October 2018, infants and young children aged 7–24 months after congenital heart disease surgery in Guangzhou were selected. Participants were categorized into two groups, in the first group (n = 95), the FFQ was completed twice at intervals of 7–10 days to assess reproducibility. In the second group (n = 98), participants accomplished both the FFQ and the 24-h diet records from 3 consecutive days to assess validity. The score of the Infant and Child Feeding Index (ICFI) and its qualified rate were caculated. Intraclass correlation coefficients (ICC) and Spearman correlation coefficient (SCC) were calculated for reliability and validity, respectively.
The average intraclass correlation coefficients and spearman correlation coefficient of the FFQ were 0.536 and 0.318, all with statistical significance except the frequency of meat added. The ICFI of the first group was 8.61 (± 3.20), the qualified rate was 0.06% (6/95). The intraclass correlation coefficients of the ICFI ranged from 0.374 to 0.958; and the spearman correlation of the ICFI was -0.066 to -0.834.
The FFQ possesses satisfactory reliability and moderate validity. The reliability of the ICFI is acceptable, but the validity results are quite different, indicating that the questionnaire is limited in the evaluation of the ICFI.
Congenital heart disease (CHD) is the most common congenital defect in neonates,with a birth prevalence of approximately 8 per 1,000 worldwide [1, 2]. CHD refers to children aged 0–3 who have anatomical changes or insufficiency in the development of heart and large blood vessels during embryonic development [1, 3, 4]. Infants with CHD often have more nutritional difficulties, including  malabsorption, decreased energy intake or increased energy damand caused by infections and/or increased metabolism . So the proportion of infants with CHD who suffer from malnutrition is higher than normal infants . Therefore, after the disease restriction is lifted, reasonable feeding is essential for the postoperative recovery and growth of infants with congential heart disease [6, 7]. However, due to the lack of nutritional status survey methods, there are relatively few studies on the diet of 7–24 months old children after congenital heart disease surgery.
Food frequency questionnaires (FFQs) are the most frequently used tool applied in large-scale epidemiological studies to assess dietary intake, as they assess usual dietary intake over longer periods of time in a relatively short time frame [7,8,9]. However, the type of food listed in each FFQ, the length of the survey period, the interval frequency, the food share evaluation method, the nutrient database, and the survey method are not identical . In addition,different groups of people have different dietary habits and geographical environment, and the accuracy of dietary intake information is closely related to the reliability and validity of the FFQ in the people who use it . Therefore,FFQ requires adoptions based on people’s dietary habits and geographic location before applying to different population [12, 13]. At present, the FFQs in China are more widely used in adolescents, youth, the elderly or special groups (such as athletes), but less for infants and young children aged 7–24 months . Therefore, it is necessary to design one FFQ for infants and young children aged 7–24 months and test the reliability and validity of it.
The Infant and Child Feeding Index (ICFI) is a comprehensive index to assess the feeding status of infants and young children, and has a certain guiding role in the feeding behavior of caregivers. The concept of the ICFI was first proposed by Ruel and Menon and primarily used for infants and young children between 6–36 months old , and they developed a multi-dimensional assessment scale with five indicators, including breastfeeding, bottle feeding or none, type of food, frequency of feeding food, general feeding frequency . Studies have also found that higher scores were associated with lower rate of malnutrition .
The aims of the study were to evaluating the reliability and validity of the FFQ for 7–24 months old infants and young children after congenital heart disease, to calculate the scores and the pass rate of the ICFI, and test the reliability and validity of the ICFI.
The study is an observational study that performed in a convenience sample of 7–24 month-old infants and young children after congenital heart disease surgery from July to October 2018 in the Guangzhou Women and Children Science Center. Participants who were followed up and checked regularly from August to October 2018 in Guangzhou Women's and Children's Medical Center were included. While the individuals with poor postoperative prognosis, combined with other severe wasting diseases, physical deformities, difficulty in feeding, and failure of guardians to complete the survey due to difficulties in understanding or filling will be excluded. This study was approved by the Ethical Committee of School of Public Health at Sun Yat-Sen University. Written informed consent was obtained from all participants.
Face-to-face interviews were conducted by a trained and experienced investigator. The FFQ and the 24-h diet records for 3 consecutive days (24 h DRs) were filled by infants' mother (caregivers). Food mold was used to help participants to estimate portion size. Participants include two parts. In the first part, the FFQ was completed twice at intervals of 7–10 days to assess reproducibility. In the second part, participants accomplished both the FFQ and 24 h DRs at the same time. The method allowing the validity to be assessed (Fig. 1).
The ICFI was introduced and improved with the growth curve of Chinese infants by Jianqiang Lai in 2012 [14, 17, 18], which contains the following terms: breastfeeding or not, feeding with bottles or not, variety and frequency of food in the last 24 h, days of adding complementary food in the past week, the first feeding time of the formula milk and complementary food.The total score of the ICFI was obtained by summing the scores of the indicators according to the 6- to 8-month-old, 9 to 11-month-old and 12 to 24-month-old categories.The total score of the ICFI evaluation index used in this study was 15 points. According to the Chinese Center for Disease Control, ICFI above 9 points was classified as qualified.
Sample size estimation
The sample size for the reliability and validity test is 5–10 times the number of "subscale" entries containing the most items in the questionnaire . In this FFQ, the maximum number of entries in the questionnaire is 9 and increased by 10 times, that means the sample size is not less than 90.
In this survey, a convenient sampling method was used to conduct the reliability and validity tests. First, number the overall survey units one by one; arbitrarily specify the starting point and order of sampling on the random number list; then draw sample unit numbers from the random number list in turn. Any number drawn within the number range is the number of the sample unit, until the number is full.Finally, 100 persons were randomly selected from nearly 300 infants and young children aged 7–24 months,and after inclusion and exclusion, the final sample size for the reliability test was 95 and the final sample size for the validity test was 98.
FFQ and 24 h DRs
The 79-food-items FFQ of 7–24 months old infants and young children was revised based on the 2016 Chinese Dietary Guidelines, the feeding guidelines for infants and young children aged 7–24 months, and the previous FFQs in China, taking into account the dietary feeding characteristics of infants and young children . The dietary questionnaire included the frequency of breastfeeding and formula feeding, the amount of formula milk every time (mL), the time of first supplementation of complementary food and formula, the frequency of complementary food supplements in the past week, the frequency and amount of fruit, vegetables, meat (chicken, duck, goose), red meat (pork, beef, lamb), fish, soy and soy products (soybean, soy milk, tofu, soy milk), solid liquid soy products, coarse grains (corn, millet, sweet potato, mung bean, sorghum, oats, buckwheat, etc.) and egg. The frequency of addition of each complementary food is divided into five categories: never, 2–3 times a month, 2–3 times a week, 4–5 times a week, 1 time or more per day.
The 24 h DRs were conducted using the multiple-pass method, developed by the US Department of Agriculture . The 24 h DRs consist of precisely recalling, describing and quantifying the intake of foods and beverages consumed in the 24 h period prior to, or during the day before the interview, from the first intake in the morning until the last foods or beverages consumed at night (before going to bed or later, in the case of those who get up at midnight and eat and/or drink something). The information should describe the type of food and its characteristics (fresh, precooked, frozen, canned, preserved),the net quantity consumed, method of preparation, commercial brands, sauces, dressings (type of fats and oils used),condiments, liquids, multivitamin supplements and food supplements, as well as the time and place of consumption (at home, away from home), etc. .
All analysis were conducted by IBM SPSS software version 25.0 (SPSS, IBM, New York, USA). In the reliability test, the correlation analysis was performed after encoding the frequency data of each level, such as frequency of complementary food supplement: 0 for "never"0, 1 for "2–3 times per month", 2 for "2–3 times per week", 3 for "4–5 times a week", and 4 for "1 or more times a day". In the validity test, the results of 24-h DRs were first coded by food classification and portion size conversion, then summed and averaged to the weekly frequency of food addition and the amount of each addition (number of servings or ml) in the same dimension as that of the FFQ; finally, the results of the FFQ and the converted 3-day 24-h DR results were coded uniformly: "never" was coded as 0 (0 times a week), "2–3 per month" was coded as 1 (one time per week), "2–3 times per week" was coded as 3 (3 times a week), and "4–5 times per week" was coded as 5 (5 times a week), "1 or more times per day” was coded as 7 (7 times a week), and the 3-day 24-h DR results were coded as integer parts of the weekly frequency.
Continuous variables are expressed as mean (Mean), median (Median) and percentile (P25, P75). The reliability test indicator is expressed by the interclass correlation coefficient (ICC) to measure the test–retest reliability, the value of the ICC is between 0–1, ICC < 0.4 indicates poor reliability, ICC > 0.75 indicates good reliability. In the validity test, the Spearman correlation coefficient was used as the index of validity test, with values ranging from -1 to 1. ρ > 0.7, ρ = 0.3–0.7, ρ < 0.3 separately means strongly correlated, moderately correlated, and weakly correlated, respectively, α < 0.05 . The total score of the ICFI in this study is 15 and the ICFI ≥ 9 is qualified according to Chinese center for disease control .
Reliability of the FFQ1 vs. FFQ2
The results of the test–retest reliability of FFQ1 vs. FFQ2 (Table 1) showed that the frequency and the portions of dietary supplements in the 7–10 days were all positively correlated. The average ICC was 0.536,ranged between 0.266 for portion of red meat and 0.904 for frequency of breastfeeding. Among them, the addition of breast milk, formula and the frequency of complementary food supplements had good reliability with ICCs were all above 0.5, the reliability ICCs for portions of various complementary foods were lower than the frequency of additions. The reliability of ICCs were all statistically significant (P < 0.05).
Validity of the FFQ vs. 24 h DRs
The frequency of meal additions and portion recorded in the FFQ and 24 h DRs did not obey the normal distribution, so spearman-related analyses were used. The results of the validity test (Table 2) showed that the FFQ was positively correlated with the frequency and portion size of meal additions recorded in the 24 h DRs, and the average SCC was 0.318(range: 0.166- 0.856),60% SCCs were above 0.3.
Among them, the frequency of breastfeeding and formula milk had good validity (SCC > 0.5), so did the frequency of fruits, red meat, egg (SCC > 0.4), however, the frequency and portion of the supplementary food had a low value of SCC. The frequency and portion of solid soy products were almost 0 so the index of validity could not be drawn. Except for the frequency of meat, the SCCs of the remaining validity indicators were statistically significant (P < 0.05).
Infant and child feeding index
The ICFI was 7.68 ± 3.69 in the samples tested for reliability and validity, and the acceptability of the ICFI was 39.9% (77/193).
The reliability and validity of the ICFI were separately calculated for all samples, and the results (Table 3) showed that the reliability ICCs were good (range: 0.374–0.958), and were all statistically significant(P < 0.01); the validity SCCs ranged between -0.066 for frequency of eggs and 0.834 for breastfeeding, the SCC of each item was less than 0.4 except for breastfeeding and nipple use, among this,only frequency of fruits (times/week) and 24 h complementary food species were statistically significant (P < 0.05).
This study evaluated the reliability and validity of the FFQ and ICFI for the infants and young children of 7–24 months after cardiac surgery for congenital heart disease in Guangzhou from July to October 2018. The current situation of the ICFI and its qualification rate were described. Our result demonstrated a satisfactory reliability and moderate validity for all food groups intakes. The reliability of the ICFI is acceptable, but the validity results are quite different, indicating that the questionnaire is limited in the evaluation of the ICFI.
The reliability test index ICC in this study was between 0.266–0.904, with an average of 0.536. Compared with the local and foreign studies, the ICC was higher than other studies [10, 24], indicating that the reliability of the questionnaire was acceptable. The result showed the reliability of breast milk and formula addition was the best, which is primarily due to that the breast milk and formula milk are the main food resources of infants and young children aged 7–24 months. The reliability of the complementary food’s portion was not as good as its frequency, because the frequency is high and stable, but the portion may be varied with the recent climate, family eating habits and preferences, etc.. It is difficult to get statistically relevant results of the beverage while 7–24 months old infants and young children drink less beverage and the Chinese infant and young children's dietary guidelines also recommend adding dairy products after 12 months old . In addition, the reliability of the questionnaire may also be affected by other factors. For instance, the duration of the time interval would affect the reliability of the questionnaire, so we adopted a 7–10 days interval consistently. Secondly, the investigator can also produce certain deviation values due to recall bias or curiosity .
The validity test index SCC in this study was between 0.166 and 0.856, with an average value of 0.318 and 60% SCC was above 0.3. Similar to local and foreign researches [10, 24], the validity of the questionnaire was acceptable. Among them, the breast milk and formula addition had good validity(SCC > 0.5), which may be related to the higher frequency of daily addition. The frequency of feedings in the 24 h DRs was significantly higher than that of the FFQ, because the frequency of feedings is unstable and easy to be affected by the infant's own condition. Among the frequency of complementary food supplements, the fruit, red meat and eggs were better reflected in the 24 h DRs and had a higher degree of coincidence with the FFQ, with SCC were all above 0.4, considering they were the most common food intake for infants and family members . The SCC of the portion of complementary food supplement was below 0.37, mainly because the 24 h DRs was not comprehensively reflecting the supplement of one-week complementary food supplement. Compared with the recalled FFQ, the 24 h DRs cannot comprehensively reflect the addition of various complementary foods for infants and toddlers in a week, which leads to a decrease in the validity, and the decrease extent of portion was greater than the frequency. In addition, it also may be that most infants were 9–11 months old, infants who at this stage have less teeth germination so they could hardly eat tough food. The poultry meat was added less in the 24 h DRs (P25, P75 is 0, the mean is 0.1), which may lead to its SCC is not statistically significant .
This study has strengths and limitations that need to be described.The 24 h DRs which selected as the "gold standard" by the validity test of this study, reflecting the real-time specific dietary addition of the respondents in three day, and had smaller recall bias and stronger training effect [28, 29]. In addition, when comparing the specific frequency indicators of the two questionnaires, such as the frequency of complementary food supplements in a week, the FFQ needs to scale the corresponding options to one week, while the 24 h DRs needs estimate the supplement status of one week from the supplement status of three days, which may be resulted in the concentration of the variety and portion of the complementary food supplements. As a result, the information loss and the estimated deviation caused by the questionnaire comparison may result in weak correlation of the two questionnaires.
The score of the ICFI was 7.68 ± 3.69, which was higher than the average level of large cities in China, similar to the results of HanZhong in Shanxi and Chengdu . The high score may be related to the good care of taken by the caregivers of the infants after congenital heart disease surgery. And the qualification rate was 39.9% (77/193), which was higher than the value of the national small and medium-sized cities,  indicating that the feeding mode of caregivers is acceptable. The reliability of the scores of each item in the questionnaire was good (range: 0.374–0.958), and the ICC score of 80% items was above 0.5, which was consistent with the reliability study of the questionnaire, indicating that the questionnaire had a good consistency of the ICFI scores. The validity index SCC was quite different (range:-0.066 to 0.834), the SCC of the breastfeeding and pacifier use were quite well which may be due to the higher frequency, followed by the frequency of fruit addition. The lower validity of other supplementary food addition items may be due to the conversion result caused by difference of the questionnaires or the difference in frequency and portion between the two questionnaires results.
The subjects of this study were only selected from infants and young children aged 7–24 months in Guangzhou in August to October 2018. Secondly, for the selection of "gold standard", the 24 h DRs is the most common, but there are certain limitations for reflecting the addition of one week’s complementary food, when the recording time is extended or other dietary survey methods are combined to test the validity. Thirdly, the items and evaluation indicators for the ICFI have not been fully unified. Fourthly, no validation with biomarkers is carried out. Finally, ten days to evaluate the reliability is a very short period and it might have had an influence in the high correlation for some groups.
The FFQ possesses satisfactory reliability and moderate validity. The reliability of the ICFI is acceptable, but the validity results are quite different, indicating that the questionnaire is limited in the evaluation of the ICFI.
Availability of data and materials
The raw data is available through this website: https://rdd.sysu.edu.cn/.
Congenital heart disease
Food frequency questionnaire
Intraclass correlation coefficients
Infant and child feeding index
Spearman correlation coefficient
- 24 h DRs:
24-H diet records for 3 consecutive days
Bernier PL, Stefanescu A, Samoukovic G, Tchervenkov CI. The challenge of congenital heart disease worldwide: epidemiologic and demographic facts. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu. 2010;13:26–34.
van der Linde D, Konings EE, Slager MA, Witsenburg M, Helbing WA, Takkenberg JJ, et al. Birth prevalence of congenital heart disease worldwide: a systematic review and meta-analysis. J Am Coll Cardiol. 2011;58:2241–7.
Sadowski SL. Congenital cardiac disease in the newborn infant: past, present, and future. Crit Care Nurs Clin North Am. 2009;21:37–48.
Clausen H, Noren E, Valtonen S, Koivu A, Sairanen M, Liuba P. Evaluation of Circulating Cardiovascular Biomarker Levels for Early Detection of Congenital Heart Disease in Newborns in Sweden. JAMA Netw Open. 2020;3:e2027561.
Jadcherla SR, Vijayapal AS, Leuthner S. Feeding abilities in neonates with congenital heart disease: a retrospective study. J Perinatol. 2009;29:112–8.
Blasquez A, Clouzeau H, Fayon M, Mouton JB, Thambo JB, Enaud R, et al. Evaluation of nutritional status and support in children with congenital heart disease. Eur J Clin Nutr. 2016;70:528–31.
Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr. 2002;5:567–87.
Mcpherson RS, Hoelscher DM, Alexander M, Scanlon KS, Serdula MK. Dietary Assessment Methods among School-Aged Children: Validity and Reliability ☆. Prev Med. 2000;31:S11–33.
Magarey A, Watson J, Golley RK, Burrows T, Sutherland R, McNaughton SA, et al. Assessing dietary intake in children and adolescents: Considerations and recommendations for obesity research. Int J Pediatr Obes. 2011;6:2–11.
Liu D, Ju H, Yang ZY, Zhang Q, Gao JF, Gong DP, et al. Food Frequency Questionnaire for Chinese Children Aged 12–17 Years: Validity and Reliability. BIOMEDICAL AND ENVIRONMENTAL SCIENCES. 2019;32:486–95.
Organization A. Methods to monitor the human right to adequate food. Methods to Monitor the Human Right to Adequate Food. 2008.
Affret A, El FD, Dow C, Correia E, Boutron-Ruault MC, Fagherazzi G. Relative Validity and Reproducibility of a New 44-Item Diet and Food Frequency Questionnaire Among Adults: Online Assessment. J Med Internet Res. 2018;20:e227.
Moghames P, Hammami N, Hwalla N, Yazbeck N, Shoaib H, Nasreddine L, et al. Validity and reliability of a food frequency questionnaire to estimate dietary intake among Lebanese children. Nutr J. 2016;15:4.
Ruel MT, Menon P. Child feeding practices are associated with child nutritional status in Latin America: innovative uses of the demographic and health surveys. J Nutr. 2002;132:1180–7.
Qu P, Mi B, Wang D, Zhang R, Yang J, Liu D, et al. Association between the Infant and Child Feeding Index (ICFI) and nutritional status of 6- to 35-month-old children in rural western China. PLoS ONE. 2017;12:e171984.
Srivastava N, Sandhu A. Infant and child feeding index. Indian J Pediatr. 2006;73:767–70.
Zhang J, Shi L, Wang J, Wang Y. An infant and child feeding index is associated with child nutritional status in rural China. Early Human Dev. 2009;85:247–52.
闫玲, 曾果, 孙要武, 李增宁, 董文兰, 潘丽莉, et al. 6~24月龄中国婴幼儿喂养指数评价方法的建立. ed, 2012:6.
杨月欣, 苏宜香, 汪之顼, et al. 7–24月龄婴幼儿喂养指南[J]. 临床儿科杂志. 2016.
AJ M, DG R, DJ B, T M, JC C, WV R, et al. The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes. The American Journal of Clinical Nutrition. Official J Amer Soc Clin Nut. 2008;88:324–32.
Castell GS, Serramajem L, Ribasbarba L. What and how much do we eat? 24-hour dietary recall method. Nutr Hosp. 2015;31(Suppl 3):46–8.
Rendo-Urteaga T, Saravia L, Sadalla CT, Monsalve-Alvarez JM, Gonzalez-Zapata LI, Tello F, et al. Reliability and validity of an FFQ for South American children and adolescents from the SAYCARE study. PUB HEALTH NUT. 2019;23:1–9.
Watson EO, Heath AL, Taylor RW, Mills VC, Barris AC, Skidmore PM. Relative validity and reproducibility of an FFQ to determine nutrient intakes of New Zealand toddlers aged 12–24 months. Public Health Nutr. 2015;18:3265–71.
Xue Y, Yang K, Wang B, Liu C, Mao Z, Yu S, et al. Reproducibility and validity of an FFQ in the Henan Rural Cohort Study. PUB HEALTH NUT. 2019;23:1–7.
Marchioni DM, Voci SM, Lima FE, Fisberg RM, Slater B. Reproducibility of a food frequency questionnaire for adolescents. Cad Saude Publica. 2007;23:2187–96.
Collese TS, De Moraes A, Rendo-Urteaga T, Luzia LA, Rondo P, Marchioni D, et al. The Validity of Children’s Fruit and Vegetable Intake Using Plasma Vitamins A, C, and E: The SAYCARE Study. Nut. 2019;11:1815.
Tabacchi G, Amodio E, Di Pasquale M, Bianco A, Jemni M, Mammina C. Validation and reproducibility of dietary assessment methods in adolescents: a systematic literature review. Public Health Nutr. 2014;17:2700–14.
Kolodziejczyk JK, Merchant G, Norman GJ. Reliability and validity of child/adolescent food frequency questionnaires that assess foods and/or food groups. J Pediatr Gastroenterol Nutr. 2012;55:4–13.
应艳红,潘建平,黄建玲等.汉中市婴幼儿喂养指数与体格发育的相关性研究[J]. Chinese Journal of Woman and Child Health Research. 2017;(28).
Association between feeding index and growth of infants and young children from National Nutrition and Health Survey on 2002. Nutrition Newsletter. 2007.
The authors gratefully acknowledge the contribution of the Guangzhou Women's and Children's Medical Center and Shenzhen Nanshan District Maternal and Child Healthcare Hospital, and also gratefully acknowledge technical support from the School of Public Health, Sun Yat-sen University.
This research was funded by Guangdong Provincial Natural Science Foundation (Grant No.2017A030313844), Sanming Project of Medicine in Shenzhen (Grant No. SZSM201803061)and the Fundamental Research Funds for the Central Universities in SYSU (Grant No.15ykpy09). The funders had no role in the design, analysis or writing of this article.
Ethics approval and consent to participate
This study was approved by the Ethical Committee of School of Public Health at Sun Yat-Sen University. Written informed consent was obtained from a parent of the participants.
Consent for publication
Written informed consent for publication of personal and clinical details was obtained from the parents.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lei, Y., Liu, Y., Hu, C. et al. Reliability and validity of the FFQ and feeding index for 7-to 24-month-old children after congenital heart disease surgery. BMC Pediatr 22, 348 (2022). https://doi.org/10.1186/s12887-022-03357-4