Background

Food- Frequency Questionnaires (FFQs) are designed to assess habitual diet by asking about the frequency with which specific food items are consumed over a reference period [1, 2]. This tool has been the most frequently used dietary assessment method in large-scale epidemiological studies and other nutritional research. Compared to other dietary assessment methods, the FFQ is easy to administer, has relatively low cost, and provides a rapid estimate of usual food intake [3]. However, investigators have recognized that nutritional values reported from FFQ data are subject to substantial error, both systematic and random, that can profoundly affect the design, analysis, and interpretation of nutritional epidemiologic studies [4, 5]. For example, it is essential to covert food composition values from an FFQ into macronutrient and micronutrient values, but a major limitation in interpreting data from FFQs is the lack of homogeneity in food composition tables. Therefore, to properly interpret the results of epidemiological studies that use FFQs, it is necessary to know the relationship between reported intakes from the FFQ and true usual intakes [6]. Multiple dietary recalls [6–8], food records [9], and biomarkers [10] are generally considered to be more accurate reference measures of nutrient intake, and thus can be used in measuring the validity of FFQs. Validation correlations vary depending upon the nutrient, but typically range from 0.40 to 0.70 [8, 11, 12].

FFQs are widely used throughout the world for epidemiologic nutrition surveys. However, due to differences in food supply and dietary habits from one population to another, there is no universally accepted FFQ that can be used for all populations. A self-administrated FFQ, used for assessing the relationship between habitual diet and Colorectal Cancer (CRC) in adult residents of Newfoundland and Labrador (NL), was developed from the well-known Hawaii FFQ [13, 14] and modified by NL researchers. Investigation of CRC in this population is warranted as NL has the highest CRC incidence rate in the Country, when compared to other Canadian provinces [15]. The diets of residents of this province have been described as ‘unique’ due to the geography, economics, culture and population demographics [16], and thus an investigation into the possible relationship between dietary factors and CRC is especially warranted in NL. It has been suggested that elucidation of diet–disease relationships requires dietary assessment methods which can adequately describe and quantify intakes, minimize systematic errors and provide reasonably precise estimates of variability between individuals and/or groups [17]. However, the developed FFQ has not yet been appropriately validated for a NL population which makes some of the findings of the CRC study difficult to interpret.

Thus, the objectives of the present study are as follows: 1) to address whether this self-administered FFQ is valid in the NL general adult population by comparison with the results of multiple 24-hour dietary recalls (24-HDRs); and 2) to provide a validated NL based self-administrated FFQ for future use.

Methods

Sample recruitment and study design

Based on the information (means and standard deviations for various nutrients) derived from the FFQ data of the on-going CRC project [18–20] and the generally acceptable correlation coefficient value of 0.6, the minimum sample size for this study was determined to be 98 participants. The validation study lasted approximately one year and each subject was contacted a minimum of three times. A 30% attrition rate per step was expected. Therefore, an initial sample size of 450 subjects was required.

During February 2011, experienced telephone interviewers recruited a random population-based sample of NL adults, aged 35–70 years, using a list of land-line telephone numbers purchased from Info Canada [21]. After excluding non-residential telephone numbers, 683 potential subjects were identified as eligible and 432 (63%) initially agreed to participate in the study. Eligibility criteria included non-institutionalized adult resident of NL for at least two years with no intent to move in the next 12 months; aged 35–70 years inclusive at the time of the intended interviews; able to speak and read English at a grade 8 level; and with no specific identified medical conditions (cognitive impairment, psychological conditions, or pregnancy).

We collected dietary intake data by telephone through a set of two variably timed 24-HDRs (one weekday and one weekend day) from each participant, which then was duplicated approximately six months later. This procedure aimed to obtain two sets of recalls (a total of four 24-HDRs) in different seasons from each subject. An FFQ survey was mailed out to all study participants in early 2012, six months after the completion of the second pair of 24-HDRs. Reminder phone calls were used to prompt participants to complete and return the FFQs.

Demographic information, including: age, gender, size of their community, marital status, employment status, level of education, and smoking habits, was collected by telephone interview. This study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the Interdisciplinary Committee on Ethics in Human Research (ICEHR) [22], Memorial University (No. 2010/11-057-ME). Verbal informed consent was obtained from all subjects.

Dietary assessment

The food-frequency questionnaire

The original Hawaii FFQ was designed to assess the typical food intake of individual males and females in a multi-ethnic Hawaiian/Southern Californian population [14]; it has been validated and widely used in the United States [23–25]. The FFQ administered in NL was modified to account for the unique food consumption habits in NL. Food items considered unusual in NL (e.g. tamales, ham hocks) were deleted or altered while some items commonly consumed in NL (e.g. moose meat, pickled meat) were added. This resulted in a list of 169 food and beverage items in the final instrument (available in Additional file 1).

The FFQ required participants to recall the number of times each food item was consumed per day, per week, per month, or rarely/never during the past 12 months. It also required participants to recall how many months of the year the food was consumed to account for seasonal variation in intake. Portion size options were given using standard measuring units (e.g. cups, tablespoons, slices) or by referring to photographs provided representing small, medium, and large portion sizes of some food items.

The 24-hour dietary recalls

The 24-HDRs were unannounced and conducted by telephone by trained interviewers. During the 24-HDR, each subject recalled and described in detail, all types and amounts of foods and beverages consumed in the previous 24 hours on two separate occasions, a weekday and a weekend day. Weekend days included Saturday and Sunday to capture food and alcohol consumption patterns which may be different from those on weekdays (Monday to Friday) [8, 26, 27]. The 24-hour period specified for the dietary recall was defined as the 24 consecutive hours between midnight on day one and midnight on the following day. To assist in estimating portion sizes of consumed foods, respondents were encouraged to view a measuring cup and measuring spoons as they completed their 24-HDR by telephone. At the end of this study, there were a total of four completed 24-HDRs for each participant.

Statistical analysis

Data analyses attempted to (1) assess completeness of the responses and (2) examine potential errors/outliers. Both are directly related to overall validity assessment.

Data entry

Amounts and specific types/brands of foods consumed were entered into ESHA Food Processor SQL, version 10.8, nutrient analysis software (ESHA Research Inc, 2010, Salem, Oregon) [28] under the guidance of a professional Registered Dietitian and/or dietetic graduate students. This software contains more than 35,000 food and beverage items. When an exact match was not available between a food consumed and an item offered in the ESHA database, a group decision was made pertaining to the proper categorization of the food item in question. The group always included at least two dietetic professionals/students. For instance, homemade bread, which is not offered in the database, is known to contain more flour, honey or other ingredients than the commercial bread. Nutrient information from one slice of homemade bread was calculated as following:

Nutrient estimate from one piece of homemade bread = 1.25 × Nutrient estimate from one piece of commercial bread

Calculation of nutrient intake

The nutrient composition of each item was obtained using the ESHA Food Processor. The nutrient composition data in the ESHA database is compiled from a variety of sources including the USDA Nutrient Database for Standard Reference, the USDA Database for the Continuing Survey of Food Intake by Individuals, the Canadian Nutrient File, manufacturers’ nutrient information, and over 1,000 additional sources of data.

Estimation of intake for a specific nutrient was conducted as following:

  1. 1.

    Within each round of 24-HDRs, each day was weighted appropriately to produce a synthetic week with the following formula:

    Mean Daily Nutrient Estimate = ( Weekend Intake × 2 + Weekday Intake × 5 ) / 7
  2. 2.

    Nutrient estimates from FFQ data were calculated using the product-sum method [1, 29]. Thus,

    Daily nutrient intake = ∑ [ ( reported consumption frequency of a food item , converted to times per day ) × portion size consumed of that food × ( amount of that nutrient in a standard serving size of that food ) ]

Validation study

Subjects were excluded if total energy intake from the FFQ fell outside the range of 500–5,000 kcal per day [1] (n = 4) or if more than one 24-HDR (n = 2) was rated as unreliable. We also excluded subjects with missing information (n = 4) from the analyses.

Means and standard deviations (SD) were calculated for nutrient intakes assessed by the 24-HDRs and FFQs. For the purpose of this study, the following nutrient intakes derived from the FFQ and 24-HDRs were compared: energy (kcal), protein, total fat, saturated fat, monounsaturated fat, polyunsaturated fat, carbohydrate, dietary fibre, cholesterol, carotene, calcium, vitamin A, and vitamin D. Paired-sample t-tests were used to determine differences between the means for energy and nutrients derived from the two dietary tools. All nutrient variables were log-transformed to improve normality and reduce skewness, and then were energy-adjusted using the residual method [30].

The relationship between the nutrient values from the FFQ, both the unadjusted and the energy-adjusted nutrient estimates, and averages of the two synthetic weeks of recalls were estimated using Pearson correlation coefficients. We also calculated de-attenuated correlations to remove the within-person variability found in the recalls [31] by using the following formula:

r t = r 0 1 + r / n

Here r t is the corrected correlation between the energy-adjusted nutrient derived from the FFQ and 24-HDRs, r 0 is the observed correlation, r is the ratio of the within- and between-person variance measured from the 24HDRs, and n is the number of replicated recalls (n = 4).

Furthermore, we categorized the distribution of energy-adjusted nutrient intakes into quartiles, and estimated the percentage of subjects classified into same, adjacent and extreme quartiles [10, 32, 33]. The Bland–Altman method [34] was also used to assess the agreement between the mean energy and nutrient intake values obtained using the two different instruments. We plotted the difference in intake between the two methods (FFQ-24HDR) against the mean intake of the two measures ((FFQ + 24HDR)/2). The overall mean difference indicated whether one method tends to overestimate or underestimate, and the limits of agreements (mean ±1.96 SD) were used to show how well the two administrations agree.

All analyses were conducted using the SAS statistical software package version 9.2 (SAS Institute Inc., Cary, NC, USA) and Statistical Package for Social Science (SPSS) software version 9.0 (SPSS, Inc., Chicago, IL, USA).

Results

Out of the 432 participants who agreed to participate in this study, 400 (93%) completed the first two dietary recalls; of these, 306 (77%) completed the second round of 24-HDRs and 210 (49%) completed the FFQ (Figure 1). After excluding those with unreliable data (n = 15), 195 subjects (153 females, 42 males) were included in the present analysis. The mean (SD) age of the 195 participants was 55.03 (8.75) years. Over half of the participants were employed (53.3%), were rural residents (56.9%), and the majority had completed post-secondary education (60.5%), were non-smokers (82.6%), and were married (78.5%). When comparing the demographic characteristics of the participants at baseline and the 1-year follow-up visit, no significant differences were observed (data not shown).

Figure 1
figure 1

Flow diagram of sample selection.

Full size image

Table 1 presents the means and respective standard deviations for energy and nutrients, derived from the FFQ and 24-HDRs. Values for energy and nutrients estimated by the FFQ were higher than those obtained using the dietary recalls, except for protein in men. Evaluation of the differences between these means showed significant differences (p < 0.05) for all the nutrients in women and some nutrient estimates in men (dietary fibre, vitamin A, vitamin D, and calcium).

Table 1 Comparison of nutrient intakes per day by Food Frequency Questionnaire (FFQ) and 24-Hour Recall (24-HDR)
Full size table

Correlations between nutrient intakes derived from the FFQs and the 24-HDRs are shown in Table 2 for men and women. The Pearson correlation coefficient for crude data varied from 0.17 (carbohydrate) to 0.40 (carotene) in women and 0.07(protein) to 0.56 (carbohydrate) in men. In both genders, adjusting for total energy intake improved the correlations in some nutrients (e.g. protein) but decreased the values in the others (e.g. polyunsaturated fat). However, adjustment for residual measurement error (de-attenuation) increased all correlations, ranging from 0.20 (polyunsaturated fat) to 0.52 (dietary fibre) in women and 0.13 (protein) to 0.61 (carbohydrate, dietary fibre) in men, with a median correlation value of 0.38 in women and 0.42 in men. Except for that of protein in men, all correlations were statistically significant with p < 0.05.

Table 2 Pearson correlations between Food Frequency Questionnaire (FFQ) estimates and weighted 24-Hour Recall (24-HDR) estimates
Full size table

Data for energy-adjusted nutrient intakes estimated from the FFQs and 24-HDRs were distributed into quartiles of intakes and cross-classified. A subject would be correctly classified if his/her energy or nutrient intakes were ranked into the same or an adjacent quartile by both methods. Table 3 presents the summary of cross-classification analysis. For women, classification of subjects into the same and adjacent quartiles ranged from 66.7% (polyunsaturated fat) to 79.1% (dietary fibre), while grossly misclassified individuals varied from 3.3% (carbohydrate, dietary fibre) to 9.1% (polyunsaturated fat). For men, the mean proportion of individuals correctly classified was 78.0%, while on average only 5.85% fell into the extreme quartile. Bland–Altman plots showed no serious systematic bias between the administration of the two instruments over the range of mean intakes (plots were shown in the Additional file 2).

Table 3 Percentage for cross-classification of energy-adjusted nutrient intakes into quartiles estimated from the Food Frequency Questionnaire (FFQ) and 24-Hour Recalls (24-HDRs)
Full size table

Discussion

A valid, comprehensive tool to measure nutrient intakes is essential to health research involving humans, especially when it is aimed at investigating the relationship between diet and diseases [35, 36]. The present study demonstrated that a previously developed 169-item self-administered FFQ is reasonably valid for dietary assessment in the general adult population of NL. We observed high agreement between the two methods investigated in quartile categorization, as more than 74% women and 78% men were correctly classified into the same or adjacent quartiles for energy and twelve nutrients. Bland-Altman plots also indicated acceptable level of agreement between the two methods.

A major component of the validation process is the selection of an appropriate reference method to test the target instrument; however no gold standard exists for dietary intake measurements. It is crucial for the errors of both the methods used in the current study to be as independent of each other as possible [37]. In a review on the validation of FFQs, Cade et al. (2002) found that 75% of the studies validated FFQs against repeated 24-HDRs [3]. The FFQ and the 24-HDRs have some similar error sources, such as the reliance on memory and the perception of portion sizes [1, 3]; however, the FFQ stresses long-term memory whereas the 24-HDR relies on short-term memory. In addition, the 24-HDR method was interviewer-based using open-ended questions, whereas the FFQ was self-administered with close-ended questions. Such differences let us assume that the errors are sufficiently independent and that the 24-HDR method is an adequate comparison method for this target instrument [38].

The present study sample was comparable to the general population with regard to geographical distribution [39]. There were significant more females than males who participated in the study. The reason may be that females are willing to care about nutrition intake and health than males [40]. Sakamaki et al. (2005) also found that females had significantly greater desire to be on a weight-loss diet than males (p < 0.001). In addition, individuals with a higher education level and those who were non-smokers were more likely to participate in the study. These may lead to potential sources of bias, which must be kept in mind when interpreting the results.

As expected, the absolute nutrient values derived from the FFQ tended to be higher than those derived from the 24-HDRs, which is a common issue reported in previous research [17, 29, 35, 41]. A possible explanation is that people tend to overestimate their actual intake when they are asked to recall the frequency of a large number of foods consumed in an FFQ [1, 29]. According to nutrient intakes of NL adults estimated in 2004 by the Canadian Community Health Survey (CCHS Cycle 2.2) [42], all nutrient intakes estimated by the current study were within the acceptable range (±20%) of the mean values.

Correlation coefficients were used to assess the association between FFQ and 24-HDRs as well as to measure the relative validity. For both genders, energy adjustment improved the correlations for the majority of nutrients. According to Willett [30], energy adjustment increases correlation coefficients when the variability of nutrient consumption is related to energy intake, but decreases correlation coefficients when the variability depends on systematic errors of overestimation and underestimation. In the present study, the lower correlation values found in some categories may indicate that the FFQ to some extent systemically over-/under- estimated intake of these nutrients, however, error in over/under estimation by the FFQ is expected. Likewise, Dehghan et al. (2012), Wang et al. (2008), and Cardoso et al. (2010) found energy adjustment did not improve the crude correlation in their studies [43–45].

Due to correction for the day-to-day variation in intakes, the de-attenuated energy-adjusted correlations were usually higher than their original values. On average, the correlation values were approximately 0.40 when genders were combined. These values are lower than some reported by previous validation studies [6, 10, 14] but comparable to others [8, 11, 46–48]. In regards to energy, lower concordance coefficients have been reported in the Willett FFQ (0.16 for women and 0.18 for men) and the Block FFQ (0.37 for women and 0.41 for men) [11] as compared with 0.26 (women) and 0.44 (men) derived from our study. It was particularly noticeable that our correlations for protein were unfavourable, especially in men (0.13), however, our findings were similar to those obtained from a Brazilian cohort (0.20) [45]. For carbohydrate in women, our study yielded a coefficient of 0.38, which compares favourably with the Jackson Heart Study (0.32) [46]. Our low correlations for polyunsaturated fat (0.20 for women and 0.26 for men) were very similar to the results of most other FFQs [11, 46–48]. This could be associated in part with xthe irregular distribution of oils used in food preparation. In terms of micronutrients, it has been suggested that the number of days which must be monitored to allow a true estimation of average daily intake is greater for micronutrients than for macronutrients and exceeds the four days investigated in this study [49]. Although our correlations for vitamin A in women (0.38) and carotene in men (0.28) were low, they were significant with p-value <0.05, suggesting reasonably good agreement between the two instruments. Other studies have also reported poor correlations for micronutrients [6, 44, 45], including vitamin A and carotene.

The use of correlation analysis for assessing validity has often been questioned on the basis that it does not measure agreement but only measures the strength of association between two variables [50, 51]. Cross-classification into quartiles of intake and Bland–Altman plots were therefore used to achieve a measure of the agreement between the two methods. In terms of total energy and all nutrients analyzed, this FFQ shows a relatively high proportion of subjects being correctly classified (into same or adjacent category) and only a small number of grossly misclassified individuals (less than 10%). As a result, we demonstrated stronger between-method agreement than other studies [52, 53]. This may reflect a high sensitivity for this instrument. Bland–Altman plots showed no systematic bias for most of the nutrients evaluated by the FFQ and the dispersion between the mean intakes estimated by the two instruments suggests a good concordance trend for some nutrients, such as dietary fibre.

Several limitations of this study must be considered. First, we did not administer an FFQ at the onset of the study, thus cannot assess the reproducibility of the instrument. Future work needs to be done to evaluate the reproducibility (reliability) of this FFQ. Furthermore, relevant information pertaining to the use of dietary supplements was not collected during the 24-HDRs. Therefore, we do not know the true nutrient intakes of this population. As well, majority participants in the present study were females, non-smokers or with higher education levels. These may result in over-or under-represented individuals in the specific groups, also may create potential sources of bias. Finally, as in most research, the general limitations of dietary assessment instruments cannot be ignored. Both the FFQ and 24-HDR methods rely on memory and may be biased due to under- or over-estimation. It has been suggested by others that multiple reference methods, including dietary methods and biochemical analyses, be used in validation studies [3, 36] to increase the accuracy of the results. Future studies may benefit from including biomarker reference methods such as urinary nitrogen and doubly labeled water; however, using a biomarker will certainly add to the participant burden and costs associated with the study. As well, it is noteworthy that use of the FFQ remains the most cost-effective way to collect usual nutrient intakes in population studies.

Conclusion

In conclusion, this 169-item FFQ developed specifically for the NL population had moderate relative validity and therefore can be used in studies to assess food consumption in the NL general population. In addition, this FFQ is capable of classifying an individual’s intake into quartiles, which is useful in examining the relationships between diet and chronic disease including CRC. Such a validation is not only immediately assisting the analyses and interpretation of data collected during the CRC study, but also contributes greatly to future epidemiological studies and other nutritional studies in NL. Further efforts should be made to evaluate the reproducibility of the present FFQ.