Background

Diet plays an important role for physical development and health status in the early life stages. Behavioural aspects contributing to disease risk in adulthood often originate in childhood and adolescence [1]. The accurate assessment of dietary intake is essential in order to investigate the relationship between diet and health [2]. Large studies and accurate methods are necessary for many nutrition research questions, but these are expensive and time consuming. Food Frequency Questionnaires (FFQs) assess the usual diet of study participants by asking the respondents about the frequency and portion size of predefined foods. In general, FFQs are time and cost efficient and have therefore become established in estimating usual food intake in population studies [3]. However, FFQs are known to have limitations and to be prone to measurement errors [4]. Especially children and adolescents have problems estimating the usual portion sizes and remembering their diet over a long time period. The reasons are, among others, unstructured eating patterns and more frequent meals outside the home [5]. Although some FFQ validation studies have been conducted for adolescents [6–10], validity in subgroups concerning age, sex, body weight, socio-economic status, etc. was not well examined [11]. Since biased results, even after stratification, can lead to wrong associations, the relative validity of FFQs should be determined by comparison with an established method in the population of interest.

A self-administered, semi-quantitative FFQ was used in the German National Health Interview and Examination Survey for Children and Adolescents (KiGGS 2003-2006) [12]. The main purpose of this questionnaire was to rank participants according to their food intake, but not to estimate the complete diet. Data are used to analyse diet-disease associations, as confounding variables within other exposure-disease associations, and to compare consumption patterns within population groups. In the 2006 EsKiMo study (Eating Study as a KiGGS Module), the detailed food consumption of adolescents was assessed by means of a modified diet history interview (DISHES) in a subsample of KiGGS participants. Furthermore, the participants were asked to complete the KiGGS FFQ a second time. Although EsKiMo was not primarily designed as a validation study, it enabled a food-group validation of the KiGGS FFQ using DISHES, a more comprehensive dietary assessment method, which was already validated for adults [13]. Since the EsKiMo module included a large representative sample of German adolescents, validity in subgroups concerning age, sex, body weight, and socio-economic status was also verified.

Methods

Study design

The KiGGS study was conducted between 2003 and 2006 by the Robert Koch Institute. It collected comprehensive, nationally representative data on the health of children and adolescents [14]. The aim of this nationwide survey was to give an overview of many relevant health aspects among children and adolescents. The survey included 17,641 participants aged 0 to 17 who lived in Germany and were registered in local population registries. Children and adolescents with a migration background were also included [15]. Special efforts were undertaken to include migrants: e.g. oversampling and translation of letters of invitation, information material and health questionnaires. The study consisted of the KiGGS core survey and five additional modules: the Iodine Module, the Nutrition Module (EsKiMo), the Mental Health Module (BELLA), the State Module "Schleswig-Holstein", the Motor Activity Module (MoMo), and the Environmental Module (KUS), which aimed to explore certain health-relevant topics in more detail. It would have been too costly and time intensive to conduct all the measurements in the total sample and would probably have reduced the compliance and response rate. In the core survey participants were enrolled in two steps. First, 167 sample points were chosen randomly, but in proportion to the size of the respective federal state and community. Within these points, persons were randomly selected, stratified by age, from local population registries. All participants were interviewed and investigated comprehensively about their health history and status, health behaviour, socio-demographic characteristics, etc. The FFQ "What do you eat?" was used to assess the usual diet. This questionnaire exists in two versions which differ only in the form of address. One was to be completed by the parents of the 1- to 10-year-olds, with questions formulated as "How often did your child eat...?". The other questionnaire was to be completed by the participants aged 11 to 17, with questions formulated as "How often did you eat...?".

The EsKiMo study was conducted from January to December 2006. The participants in EsKiMo were randomly selected from the KiGGS sample and stratified by age and sample point. The rationale was that about one hundred boys and girls were chosen per age group for statistically sound analyses. The validation was conducted with dietary data from 1,272 adolescents aged 12 to 17 years. The FFQ was sent to the EsKiMo participants by post three to four weeks prior to a local visit for the more comprehensive and detailed diet history interview (DISHES interview). Both instruments therefore cover largely the same time frame. The seasonality of diet was reflected at the group level by the equal distribution of the assessment over the year. Food consumption data from both methods were converted to mean intakes as grams per day, and food groups from the DISHES interview were aggregated to food groups comparable to those of the FFQ. The survey was approved by the German Federal Data-Protection Office and by the Ethics Committee of Charité University Medicine (university hospital). Respondents were informed in detail about the study objectives, interview and examination procedures, as well as the handling of data records and analysis under pseudonymous conditions, and gave their written consent. Design and methods are described in detail elsewhere [14, 16].

Dietary assessment

The self-administered FFQ "What do you eat" was developed at the Robert Koch Institute to assess the usual intake of several food groups in the KiGGS core survey (2003-2006). The food groups most often consumed by children and adolescents were selected based on data from previous surveys and the advice of nutrition survey experts [17]. Questions on the frequency and the amount of 45 food items consumed "during the last few weeks" were included. Additional questions related to specific nutritional demands (multivitamin tablets, convenience foods, light products). The frequency of consumption was assessed using ten response categories: never, once a month, two to three times a month, once or twice a week, three to four times a week, five to six times a week, once a day, two to three times a day, four to five times a day, more than five times a day. In addition, participants were asked to indicate the portion size of the food items, which was given in five item-specific categories. Several pictures were used to illustrate portion sizes. The time frame "during the last few weeks" for the FFQ was based on pre-test experience, since some participants reported that it was difficult to give an answer for exactly "the last four weeks". However, the predefined answer categories for the frequency of consumption imply a time frame of about four weeks, since the lowest frequencies relate to a frequency per month (once a month, two to three times a month). The FFQ and a covering letter were sent to the respondents by postal mail three to four weeks prior to the visit. The first page of the FFQ provides instructions on completing the questionnaire. During the survey period a telephone hotline offered support with completing the questionnaire. Furthermore, support was offered when questionnaires were collected on local visit for the DISHES interview. The development process and design are described in detail elsewhere [17].

The DISHES interview is a modified diet history interview for assessing the usual dietary intake, with a reference period of the last four weeks. This was used as reference instrument. The DISHES software facilitates a standardized, structured and interviewer-guided assessment. The procedure has a meal-based structure similar to many 24-hour recall instruments. It is standardized, but still open-ended and allows the assessment of all possible food items in detail. The DISHES interview was conducted by trained nutritionists at the residence of the participants. First, usual meal patterns were obtained. In the next step, food intakes consumed during each meal were assessed by a check list. Subsequently, the frequency and portion size of each food consumed at the different meals was determined in detail. Additional food items could be chosen by searching the food code database. In general, estimation of portion sizes was facilitated using standardized tableware models. In addition, a picture book adapted from the EPIC-SOFT Picture Book [18] could be used to determine the portion size of selected food items. The DISHES software codes food items and connects the codes with the German Food Code and Nutrient Database (BLS II.3), which includes 10,654 food codes [19]. For the EsKiMo study, the software was adapted for the target group of adolescents (DISHES Junior). Additional foods (1,225 food codes), not yet available in the BLS but often consumed by adolescents, were incorporated into the database. The average duration of an interview in the EsKiMo Study was 49 minutes. The instrument had been previously validated for adults [13] and used in several national nutrition surveys [16, 20, 21].

In the KiGGS study (2003-2006), the parents were asked about their income, occupational status and education. This information was used to calculate a family socio-economic status index, developed for the survey. The index was categorized into low (3-8 points), medium (9-14 points) and high (15-21 points) [22]. According to this index, 27.5% of the KiGGS participants were allocated to the low, 45.4% to the medium, and 27.1% to the high socio-economic status group [23]. Furthermore, the body weight and height of the adolescents was assessed by standardized measurement. The body mass index (BMI) was calculated from body height and weight. According to the Kromeyer-Hauschild method, participants with a BMI above the 90th percentile of the age- and gender-specific reference values were categorized as overweight [24].

Data and statistical analysis

The 45 FFQ items were aggregated to 40 food groups to enable a direct comparison of the two instruments. The FFQ items "fresh fruits" and "tinned fruits" were aggregated to fruits, and "cooked", "frozen", "tinned" and "raw vegetables" were aggregated to vegetables, since the original differentiation is not provided within the DISHES data. Furthermore, chocolate was added to the sweets group. Food frequency data were recoded into times of servings per month (one month being defined as 28 days). The arithmetic mean was used for frequency bands, and the frequency "more than five times a day" was set to six times a day. Portion categories were converted into gram amounts using predefined standard portion sizes. The average food-group intake was calculated by multiplying the frequency and portion size. For further information on the recoding of frequency and portion-size data, see Additional file 1. Food-level data were converted using SAS version 9.2 (SAS Institute, Cary, NC, USA). For most food groups, the food consumption was not normally distributed. Nonparametric Spearman rank-correlation coefficients were therefore calculated. Correlation coefficients were calculated for all participants and stratified by sex, age group, BMI, and socio-economic status. The commonly desired outcome from an FFQ is a good-ranking capability of participants [25]. To evaluate the agreement in ranking, participants were grouped into quartiles for each food group. Construction of quartiles was impossible for food groups where more than 25% of participants reported no consumption. Zero consumers were therefore defined as one group and the remaining participants grouped into tertiles. This was necessary for the following 20 food groups (percentages of zero consumers FFQ;DISHES): sport/energy drinks (64%;92%), tap water (46%;82%), fruit/herbal tea (33%;58%), black/green tea (73%;88%), coffee (69%;74%), breakfast cereals (17%;33%), brown bread (11%;39%), butter (31%;41%), margarine (46%;42%), cream cheese (37%;59%), eggs (13%;26%), fish (23%;33%), pasta/rice (0%;28%), cookies (17%;35%), ice cream (12%;30%), cream desserts/pudding/rice pudding (29%;41%), pancakes (29%;55%), sweet spreads (23%;37%), hazelnut spread (28%;44%), and nuts (52%;70%). Classification into the same, adjacent and opposite quartile or group was subsequently calculated. In addition, the degree of agreement was evaluated with the weighted kappa coefficient (κw) using the formula [26]:

κ w = O w - C w 1 - C w

For this, a cross table (4 × 4) of frequencies was calculated for each food group. The observed proportion of agreement (Ow) and the expected proportion of agreement by chance (Cw) were derived. The weighting factors were 1 for complete agreement (same quartile), 0.66 for persons differing in one category (adjacent quartile), 0.33 for persons differing in two categories, and 0 for complete disagreement (opposite quartile). The mean intakes derived from the FFQ and the mean differences between both methods were calculated according to the formula: Mean of difference = Mean (FFQ - DISHES). Furthermore, the mean % of difference was calculated according to the formula:

Difference ( % ) = Mean ( FFQ - DISHES ) ( Mea n FFQ + Mea n DISHES ) ∕ 2 * 100

The 2.5 percentiles and the 97.5 percentiles of the difference were calculated. This represents the range of 95% of all differences. All statistical analyses were performed using SPSS version 18.0 (SPSS Inc., Chicago, Illinois, USA). Non-overlapping 95% confidence intervals were considered statistical significant.

Results

The present analysis included 1,249 EsKiMo participants, who completed both instruments (FFQ and DISHES interview). Within the core KiGGS study, participants were excluded from the analysis of FFQ data if they reported having consumed over six litres of beverages and over four kilos solid food, or if there were more than 20 food items missing. Since the validation study is primarily for the evaluation of KiGGS, we used the same criteria. Thirty respondents had too many missing values for frequency questions and were excluded from the validation. Three respondents were excluded because of implausibly high consumption data. Finally, the sample for the statistical analysis included 1,213 adolescents aged 12 to 17. The characteristics of the validation sample are shown in Table 1. The sample includes 582 boys and 631 girls.

Table 1 Characteristics of study participants
Full size table

Correlations

Table 2 shows the correlation between the two methods in different food groups. The correlation coefficients for the total group of participants varied between 0.22 (pasta/rice) and 0.69 (margarine); most values were 0.50 and higher. Correlation coefficients between 0.3 and 0.5 were observed for potato products, pancakes, meat, vegetables, cakes/pastries, tap water, cookies, poultry, nuts, bread and sport/energy drinks. Only for the food group pasta/rice was the correlation coefficient less than 0.3.

Table 2 Correlation coefficients (95% CI) of food group intake between both methods by sex and age*
Full size table

Subgroups

Correlation coefficients were similar for boys and girls in most food groups (Table 2). Nevertheless, significant differences between the sexes were observed in three food groups. The correlation coefficients for vegetables and sport/energy drinks in the female study group were significantly lower and the correlation coefficient for poultry significant higher than in the male group. Compared to younger participants, adolescents aged 16 to 17 showed a tendency to higher correlation coefficients. Significantly higher correlation coefficients were observed for 16- to 17-year-olds than for younger adolescents for fish and white bread. In addition, 16- to 17-year-old adolescents had significantly higher coefficients for coffee and breakfast cereals than 12- to 13-year-olds, and higher coefficients for fast food than 14- to 15-year-olds. Table 3 shows the correlation coefficients between the mean daily food intake assessed with the FFQ and the DISHES interview stratified for relative bodyweight (normal weight, overweight) and socio-economic status (low, medium, high). Coefficients for overweight adolescents were lower than those for adolescents with normal body weight in most cases. Significant differences were observed in the case of fruit/herbal tea, butter, cream cheese, meat products, cakes/pastries, sweets, and hazelnut spread; correlation coefficients were higher among normal-weight respondents. After additionally stratifying for sex, a tendency towards lower correlation coefficients was observed among overweight girls compared to normal-weight girls, while overweight boys often showed higher coefficients than normal-weight boys (see Additional file 2, Table S1). A comparison between adolescents with low and high socio-economic status showed a tendency towards higher coefficients for adolescents with higher status (Table 4). Significant differences between these groups were found in the case of milk, fruit/herbal tea, breakfast cereals, meat products, potatoes, fast food, ketchup/mayonnaise, and cakes/pastries; correlation was higher for high socio-economic status. A further stratification for sex showed similar results for boys and girls (see Additional file 2, Table S2).

Table 3 Correlation coefficients (95% CI) of food group intake between both methods in subgroups*
Full size table
Table 4 Mean of food intake assessed by the FFQ and mean of difference between both methods
Full size table

Ranking classification

Table 5 presents the agreement between quartiles of food consumption from the FFQ and quartiles from the DISHES interview. The proportion of participants classified in the same and adjacent quartile varied between 70.1% for pasta/rice and 90.8% for coffee. Classification in opposite quartiles varied between 1.9% for soda/mineral water and 9.7% for tap water. For most food groups Cohen's weighted kappa showed values between 0.21 and 0.60. Only the food groups white bread and pasta/rice showed values below 0.20.

Table 5 Agreement of quartiles for food group intake assessed by both methods
Full size table

Mean differences

Table 4 shows the mean food-group intakes per day estimated by the two methods and the 2.5 and 97.5 percentiles of the differences. The mean difference ranged from 1.4% for milk to 100.3% for pasta/rice. Milk, mineral water, eggs, meat, fish, fruits and potato products showed differences of less than 10%. Food consumption as assessed by the FFQ was not generally higher or lower than the consumption estimated by the DISHES interview. The intake of soda, juice, mineral water, fruit/herbal tea, coffee, breakfast cereals, white bread, butter, margarine, meat products, fish, vegetables, fast food, ketchup/mayonnaise, cookies, sweets, pudding/rice pudding, sweet spreads, hazelnut spread, and nuts assessed by the FFQ was lower than the estimates by the DISHES interview. The intake of milk, sport/energy drinks, tap water, black/green tea, brown bread, cheese, curd, cream cheese, eggs, soup, meat, poultry, fruits, pasta/rice, potatoes, potato products, cakes/pastries, ice cream, pancakes, and salty snacks was higher. The 2.5 percentiles and 97.5 percentile of differences covered a wide range.

Discussion

In the present study, the validity of the KiGGS FFQ was evaluated in comparison to a diet history method instrument. Due to measurement errors and limitations within every dietary assessment method, only relative validity can be determined. The FFQ showed a fair to moderate agreement in ranking participants towards their intake for most food groups compared to the DISHES interview [27]. Only white bread and pasta/rice showed slight agreement. The correlation coefficients varied between 0.22 for pasta/rice and 0.69 for margarine. A reasonable to good correlation between the two instruments was found for 67% of the food groups [28]. The average of the observed correlation coefficients was higher or equal to other FFQ validation studies for adolescents [6–9, 29–32]. The observed correlation coefficients are also similar to results from FFQ validation studies for adults [33–36]. Individual, higher coefficients for adults may be caused by an established meal structure and therefore a better memory on portion size and frequency. By contrast, the food frequency and portion sizes of adolescents are not constant [37]. Agreement of mean intake is rather low in most food groups. Some food groups - like milk, mineral water, eggs, meat fish, fruits and potato products - show small average differences. However, on the individual level there is a wide range of differences for every food group. The FFQ should therefore perhaps not be used to estimate absolute intakes. Other youth validation studies on food group level came to similar results [9, 29].

The validation was performed using food consumption data from the EsKiMo module. This offered the advantage of a large validation sample that is representative of German adolescents, which also made it possible to evaluate the validity in subgroups. However, there may have been a tendency to select participants who were especially interested in their health and nutrition, since the EsKiMo participants agreed to participate for a second time. Calculation of correlation coefficients is a common method in validation studies [38]. One main reason may be that it facilitates comparisons with other study results [39]. However, correlation coefficients only measure the strength of the association between two methods, not the agreement, and can be a misleading indicator of validity [40, 41]. Nevertheless, calculating correlation coefficients was included in this study since small correlation coefficients can be indicators of potential error sources [42]. Additional analyses, like Bland Altman analysis or ranking classification, can avoid misleading conclusions. For the Bland Altman analysis it is assumed that the differences between two measurements are normally distributed [43]. Since in our study the differences were not normally distributed, and this could not be improved by log-transformation, the differences between the two instruments were calculated on the basis of untransformed data. Furthermore, we included an adapted analysis, which approximates the analysis of limits of agreement. Percentiles (2.5/97.5) of differences between the methods were calculated, which also represents 95% of differences. There are some limitations to be considered in relation to this validation study. For the assessment of validity, the reference method should have independent error sources [44]. Contrary to this, the reference instrument DISHES also relies on the memories of the participants and their perceptions of portion sizes, like the FFQ. This may result in unrealistically higher estimates of validity. Since the EsKiMo study was not primarily designed as a validation study, the choice of another reference method was not applicable. However, the DISHES interviews were conducted by trained nutritionists and supported by standardized software, while the FFQ was self-administered. Dietary intake information was more detailed and assessed in a meal-based structure. The DISHES interview used a variety of tableware models, standard portions and a picture book for estimating portion sizes. Furthermore, the list of food items assessed by the FFQ was fixed, while the DISHES interview was open-ended. The DISHES interview therefore seems an acceptable method of comparison. The DISHES method was previously validated for adults, but not for adolescents. It has also been used in some large nutrition surveys. Nevertheless, a pre-test was conducted to test feasibility among adolescents and the food-code database was adopted for younger persons. The FFQ was filled in by respondents several days before the DISHES interview was conducted. The sequence of instruments is relevant, since one measurement may affect a later response [44]. However, the reverse sequence would probably have a larger effect, since the diet history is a more comprehensive instrument which may have a larger impact on a person's memory and awareness of the actual diet. In addition, the items in DISHES are more detailed and asked in a face-to-face setting. We therefore think the influence on recall of the applied sequence is minor.

The FFQ seems to be suitable for all considered subpopulations, since most food groups showed reasonable to acceptable correlation coefficients. Only the groups pasta/rice, white/brown bread, and cakes/pastries showed correlation coefficients of below 0.3; these should be interpreted with caution. Despite these results, certain differences between the BMI and socio-economic groups were found. As expected, older participants (aged 16 to 17 years) showed a tendency towards higher correlation coefficients than younger ones (12 to 13 years), because their cognitive abilities were better developed [37]. Furthermore, older adolescents choose their food themselves more often; they are also more conscious of what they eat. Correlation coefficients were lower for overweight adolescent girls compared to normal-weight girls. This finding might be expected, since thinness and body image have an important influence on female adolescents' dietary reporting [45]. Boys are less likely to be concerned about their body image. This relationship is in line with results from other studies [46, 47]. In addition, participants who live in families with a low socio-economic status showed lower correlations more often than participants in families with high socio-economic status. To our knowledge, similar studies in such subgroups have not been performed among adolescents. However, some studies among adults found an inverse association between socio-economic status and underreporting [48, 49], which is a potential source of bias in nutritional epidemiology and could be one reason for lower reporting validity. Nevertheless, the difference between subgroups is marginal and most food groups showed acceptable to good correlations. The KiGGS-FFQ is thus also suitable for groups with lower socio-economic status and higher body mass index.

Some differences in ranking and mean estimates between the instruments may be caused by differences in the measurement of portion sizes. While the DISHES interview assesses food intake with a variety of standard portions, tableware models and a picture book, the FFQ uses predefined, simple categories. The variability of values measured by the FFQ is therefore rather low. The relatively weak ranking agreement in the case of vegetables is discussed in other studies among adolescents [9, 29], and also in adult populations [33, 35, 50]. One possible explanation is again related to difficulties in estimating portion size in some food groups. Some broad food items like "cooked", "frozen", "tinned", and "raw vegetables" may complicate the estimation of these predefined portion sizes. For instance, lettuce and tomatoes both belong to the raw vegetables group, even though one portion of each may have very different weights. In addition, adolescents in particular may have problems defining the origin of their foods, because they normally do not prepare meals themselves. Accordingly, dividing vegetables into the groups frozen and tinned seems difficult for this age group. This difference was not assessed in the DISHES interview. These items were therefore grouped. The food group pasta/rice showed slight agreement among ranking participants in terms of food intake. One possible explanation is the different use of the two products. Pasta is often the main component of a meal like spaghetti bolognaise, while rice is eaten as a side dish. It is therefore difficult to predefine a portion size for both products together. The DISHES interview assesses the amounts separately for every food and in as much detail as possible. The group white bread also showed only slight agreement in ranking. This may be due to a lack of experience among adolescents regarding the classification of bread.

Conclusions

The FFQ shows fair to moderate ranking validity for most food groups except pasta/rice and white bread. Estimates for these two food groups should be interpreted with caution. As for the complete diet, the ability to assess absolute intakes using the FFQ is limited; but also for single foods there is no evidence whether the data of the DISHES interview or the FFQ are closer to the truth. Overall, the relative validity of the KiGGS FFQ is comparable to FFQs from the current literature [9, 29, 33, 35, 36]. The FFQ seems suitable for collecting representative dietary data at the population level, which allows exposure comparison and confounder adjustments. Based on correlation coefficients, the validity is similar for age, sex, body weight, and socio-economic status subgroups.