Introduction

Cardiovascular disease (CVD) is one of the leading causes of death and disability globally. In the past 10 years, the global mortality rate of CVD has increased by 12.5% [1]. CVD is characterized by high morbidity, disability, mortality and recurrence rates [2]. According to the Global Burden of Diseases 2017, stroke and ischemic heart disease are the top two causes of death in China. Accounting for stroke and ischemic heart disease of deaths annually is 149 and 124 per 100,000 population [3]. Dietary risk factors have been found to influence the occurrence, progression and mortality of CVD [2, 4, 5]. Reasonable dietary intakes including Mediterranean Diet supplemented with extra-virgin olive oil and Mediterranean Diet supplemented with nuts can reduce the occurrence of CVD by 50% [6]. Many epidemiological studies have also shown that appropriate dietary habits play a very important role in the protection and prevention of CVD and adapting appropriate dietary patterns such as diets high in monounsaturated and polyunsaturated acids that favor metabolic markers may prevent CVD [7]. On the contrary, malnutrition may lead to an increase in morbidity and mortality. Considering that nutrition is an important modifiable risk factor for people at high risk of CVD, it is important to understand the current dietary intake [8]. Therefore need to have appropriate assessment tools for dietary patterns [9].

In nutritional epidemiology, there have been numerous tools to assess dietary intake and each method has its own advantages and limitations [10,11,12]. The most commonly used assessment tools are the 24-hour dietary recall and dietary records containing weighing foods are also used to measure daily dietary intake, but these methods are expensive, time-consuming, and not suitable for most large-scale studies [13]. Besides, short-term recalls and diet records are not representative of usual dietary intake. Therefore, it is not sufficient to assess dietary intakes over time. Food Frequency Questionnaire (FFQ) is considered to be a low-cost diet assessment method suitable for a large sample and is often used to research the relationship between dietary factors and diseases in various epidemiological studies [14, 15]. Due to its long reference period and pre-specified food list, the collected dietary intake information has limitations to accurately measure the dietary intakes [16]. In other words, the nutritional content of FFQ is mainly affected by systematic errors and requires a careful evaluation when assessing diet-disease relationships [17, 18]. Also, the differences in demographic, socio-economic, cultural, and other points also influence the food intake of each group [19, 20]. Therefore, it is necessary to verify the validity and reliability of the FFQ in each group to evaluate the accuracy and precision of dietary intake.

A recent study evaluated the effectiveness of an SFFQ in a group with CVD [18]. The results showed that due to the short time span of SFFQ, there were large seasonal differences in nutrient estimates. Thus, there are some limitations in the research. In addition, the result cannot be transferred to other groups in different dietary regions, and each region should develop localized SFFQ based on its specific dietary habits and traditions rather than using a uniform SFFQ. The mortality rate of CVD in Ningxia residents showed an increasing trend from 2012 to 2016 and was mainly over 60 years old [21]. At the same time, the unreasonable dietary patterns of adult residents in Ningxia was manifested by insufficient intake of vegetables and fruits and excessive intake of salt and edible oil, which can increase the risk of related diseases [22]. We were interested in the consumption of polyunsaturated fatty acids in this population. Because the program was a long-term dietary intervention, FFQ was a relatively good dietary survey method.

Therefore, the aim of this study was to assess the validity and reliability of an SFFQ used to assess dietary intakes in groups at high risk for CVD in Ningxia.

Methods

This study collected two FFQs over6 months. The first FFQ (FFQ1) was collected in December 2019 and the second FFQ (FFQ2) was collected in July 2020. Two discontinuous 24-hour dietary recalls were collected, at the same time as FFQs. All the dietary interviewers were conducted on a random day because pre-trial tests showed there was no significant difference between weekdays and weekends in the diets of residents in the areas.

Study populations

Participants have been recruited from six villages from Qingtong Xia County of Ningxia province in China and they were same as the participants of project [23]. Based on the study of Willett [10], at least 110 participants were required to examine validity and reliability of dietary surveys. The study recruited 210 participants at high risk of CVD, defined as having a history of CVD. In each village, A history of CVD or a high risk of CVD, defined on the basis of a prior hospitalisation or male aged> 60 years, or female aged> 65 years, and with at least two of the following risk factors: a. Type 2 diabetes requiring treatment with at least two oral anti-hyperglycaemic agents and/or insulin b. Systolic blood pressure > 140 mmHg while on one or more antihypertensive agents c. Current daily smoking. Participants with an intellectual disability, a cognitive disability or with any chronic medical condition which required dietary restriction were excluded. The doctors will inform those participants to prepare relevant supporting documents (medical records and physical evidence of cigarettes and drugs) prior to participating in screening interviews.

The study was reviewed and confirmed by the Ethics Committee of Ningxia Medical University (No. 2020–066).

The SFFQ

The SFFQ was developed from a pre-trial of 20 people, conducted by household interviews. In the pre-trial, food pictures were used to help recall food types. There were two standardized trained staff on site. The SFFQ was formulated by combining the types of foods that were based on the 3 days 24-hour dietary recalls survey in local household. The questionnaire concluded 173 types of typical foods from areas of Ningxia, including foods unique to northwest China and accounting for about 95% of the most commonly used foods in Ningxia. Participants had the option to type name of foods if not in the SFFQ. These food items were allocated to 15 food groups such as: cereals, beans and soy products, potatoes, milk and dairy products, meat and meat products, eggs, aquatic products, vegetables, fruit, the fungus mushrooms, oils and fats, alcohol, beverages, snacks and condiments. For each food item, four categories were provided for frequencies (daily, weekly, monthly, annually or never) and the amount of consumption over the past 6 months was reported using the common weight unit in China (1 liang =50 g) [24]. The monthly consumption of the whole family was used for estimation of oil and condiments consumption. The SFFQ was designed to survey the dietary intakes of participants over the past 6 months and was administered by two staff with standard training due to the low educational level of the participants.

The 24-hour dietary recall

All recall interviews were conducted in the subjects’ homes to estimate commonly used home measurements more accurately and to limit the number of missing subjects. Participants were asked to recall all the foods or beverages they had eaten in the previous 24 hours and estimate the portion size. Common household measurements (the size of bowls and plates) are used to help estimate portions. Other dietary information includes recipe ingredients, the measures of cooking and the time and place of they had eaten (e.g. home or outside). Mixed foods in the 24-h dietary recall are converted to their ingredients to measure.

Calculation of nutrient intake

The data obtained from the FFQ was imported with EpiData software by two trained staff. The data from the 24-hour dietary recalls were inputted into the Nutrition Calculator (v2.7.5(k), Institute for Nutrition and Food Security, Chinese Center for Disease Control and Prevention). The daily nutrient intake of participants was calculated using the China Food Composition Database. The nutritional values of each food were calculated by matching the food names in the database. If there is one food not listed in the database, the nutritional values were calculated using foods that contain similar ingredients. The data of food intake and nutrient intake output by the dietary software were imported into Microsoft Excel for statistical analysis.

Statistical analysis

To measure the consistency of the first and second FFQ received by the same subject at different times, Spearman’s rank correlation coefficient (r) was calculated to compare the energy, nutrients and food intakes between two FFQ administrations. For reliability analysis, we used the inter-class correlation coefficient (ICC) and 95% CI for each point. Weighted kappa (k) statistic, and misclassification (quartiles method) analyses also were used to assess the reliability between FFQ1 and FFQ2. Among them, k values above 0.80 indicate very good agreement, 0.61 to 0.80 indicate substantial (good) agreement, 0.41 to 0.60 indicate moderate agreement, 0.21 to 0.40 indicate good agreement, and 0 to 0.20 indicate slight (poor) agreement [25].

The overall raw data were natural-log(ln) transformed to improve the normality of food groups and nutrients. Spearman’s rank correlation coefficient (r) of natural-log(ln) transformed values were calculated to evaluate the validity of FFQ and 24-hour dietary recalls. The FFQ and 24-hour dietary recall data were the mean of the two times. Energy-adjusted nutrients intakes estimates were obtained by the residual method [10]. All validity coefficients were attenuated due to random errors in the 24-hour dietary recalls. This formula from Willet was used to calculate the de-attenuated correlation to eliminate within-person variability in 24-hour recalls [10]: \({r}_t={r}_0\sqrt{1+r/n}\), where r0 is the observed correlation between FFQs and 24-hour recalls, and r is the rate of variation within- and between-person measured during two 24-hour recalls, and n is the number of days of dietary recall (n = 2). Bland-Altman plots were used to test the consistency of the two dietary assessment methods between different intakes. As suggested by Bland&Altman [26], the natural-log(ln) transformed was performed to narrow the 95% limits of agreement (LOA).

IBM SPSS Statistics Version23.0 was used for all data analysis. All p values were double-tailed, and the p values less than 0.05 were considered evidence of a statistically significant correlation.

Results

Among the 210 selected participants, 205 agreed to take part in the study and completed the survey (response rate = 97.6%). The main reasons for not participating in the study included refusal, absence during the investigation period, poor health and death. Table 1 lists the sociodemographic and anthropometric characteristics of the study population. The mean age of the subjects was 65.3 years old (male 46.3%), the mean height was 160.4 cm, the mean weight was 68.1 kg, and the mean body mass index was 26.4 kg /m2.

Table 1 Characteristics and anthropometric measurements of the participants included (N = 205)
Full size table

The median intakes of total energy, nutrients, and food groups derived from the two FFQ and the percentage of differences are presented in Table 2. The median intakes for all nutrients and food groups (except fruit) evaluated with FFQ2 were higher than or equal to the median intakes with FFQ1, with differences between 0 and 52.4%. The largest differences between the median intakes of the two FFQ were 36.0% for nutrients (cholesterol) and 52.4% for the food group (eggs).

Table 2 Comparison of nutrients intakes between two FFQs for 205 participants (median and 25th - 75th percentiles)
Full size table

The ICCs of food groups ranged from 0.34 to 0.73 and nutrients ranged from 0.25 to 0.72 (Table 3). The energy-adjusted ICCs of food groups ranged from 0.26 to 0.73 and nutrients ranged from 0.26 to 0.71. When the food groups and nutrient intakes were divided into quartiles, the agreement rates of FFQ1 and FFQ2 in the same and adjacent quartile were 72.2 to 85.9%. Except for vitamin A(10.2%), the misclassification for all nutrients and food groups as extreme quartile was rare (< 10%). Most the weighted k statistics were moderate conformity, ranging from 0.40 to 0.54. Weighted k statistics for the five food groups and seven nutrients were general conformity, ranging from 0.29 to 0.39. Only vegetables showed poor consistency, with a low weighted k statistic (0.19).

Table 3 Reliability of food groups and nutrients intakes between FFQ1 and FFQ2
Full size table

Overall, the median daily intakes of nutrients, assessed by two FFQs average, were substantially higher than the average of the two 24-hour dietary recalls, except for protein, carbohydrates, and vitamin B1 (Table 4). The unadjusted Spearman correlation coefficients for nutrients ranged from 0.18 for vitamin A to 0.81 for vitamin E. After energy adjustment, Spearman correlation coefficients of all nutrients decreased except vitamin E, which increased slightly. Compared with the unadjusted values, niacin (0.59 ~ 0.27) and iron (0.45 ~ 0.14) had the most significant changes. Except for vitamin A, vitamin D, folic acid, and iodine, the energy-adjusted coefficients of the other nutrients were statistically significant (p < 0.01 or p < 0.05). The average of correlation coefficients improved from 0.32 to 0.39 after energy adjustment, and correction for random within-person variation and between-person variation. The de-attenuated correlations for all the nutrients increased and ranged from 0.07 for folic acid to 0.98 for vitamin E. When nutrients intakes were divided into quartiles, the agreement rates between FFQs and 24-hour dietary recalls of the same and adjacent quartiles were 71.2 to 94.1%. The misclassification for all nutrients as extreme quartile was rare (< 10%), except for vitamin D (12.7%) and iodine (10.2%).

Table 4 Validity of nutrients intakes between FFQs and 24HDRs
Full size table

The results of the Bland-Altman plots showed in Fig. 1 including energy, protein, fat, and carbohydrates (Other nutrients were listed in Annex). According to Fig. 1, most of the points fell within the 95% limits of agreement (LOAS), closer to the middle horizontal line and there was no linear trend between the differences and means for two FFQs and 24-hour dietary recalls.

Fig. 1
figure 1

Bland-Altman plots for nutrients between FFQs and 24HDRs to assess the intake of: (A) protein, (B) energy, (C) carbohydrate, and (D) fat. The limits of agreement (dotted line) indicates the 95% confidence interval(mean ± 1.96SD)

Full size image

Discussion

This study aimed to examine the validity and reliability of SFFQ used to evaluate various food groups and nutrients in dietary intakes assessment of high-risk populations of cardiovascular disease in Ningxia. In this study, two discontinuous 24-hour dietary recalls were collected to evaluate the reliability and validity of the SFFQ at 6-month intervals. We assessed the performance of SFFQ by comparing the food groups and nutrients intakes reported using the instrument with intakes obtained using the 24-hour dietary recalls. Overall, the SFFQ has shown acceptable reliability and validity in evaluating food groups and nutrient intakes in people at high risk for CVD.

In previous studies [27, 28], the intervals between FFQs varied. Reliability is based on the results of diet between two questionnaires, ideally the shorter the interval, the better the reliability [10]. In this case, however, the subjects were more likely to remember and repeat their previous answers. In this study, there was a six-month interval between the two FFQs, which could relatively reduce the above bias. Similarly, for validation studies, it is crucial to select the appropriate reference method to evaluate FFQ. Previous studies have shown that FFQ and 24-hour dietary recalls have been validated in 75% of studies. Although FFQ and 24-hour dietary recall are both retrospective questionnaires, they are prone to the same recall bias, which may lead to overestimation of the correlation coefficient. However, 24-hour dietary recalls may be more appropriate for subjects with lower educational levels [20, 29]. Since participants in this study were high-risk populations in areas with limited literacy, the 24-hour dietary recall was selected for assessment.

In the reliability studies, results showed that the median intakes of all nutrient and food groups (except fruit) assessed with FFQ2 were higher or equal to the median intakes of all nutrients and food groups assessed with FFQ1. It could be that participants were more likely to pay attention to their dietary intakes after the first FFQ, or it could be that over a longer span of time, the dietary of participants changed. The ICCs of most food groups and nutrients ranged from 0.53 to 0.73. After energy adjustment, most ICCs are 0.49 to 0.73. Other reliability studies have reported similar ICC [30,31,32,33,34]. When the food group and nutrient intake were divided into quartiles, the effective quartile coincidence rate between two FFQs was high (ranged from 72.2 to 85.9%). It is rare for all nutrient and food groups to be misplaced in the extreme quartile. Most of the weighted k values have a moderate degree of conformity, only vegetables showed poor consistency. The percentage of participants correctly classified into the same or adjacent categories was slightly higher than reported in other confirmatory studies, while the weighted K values were similar to those reported in other studies [35,36,37]. In addition, our study collected FFQ data on household condiments commonly used, which made this dietary data more comprehensive than those reported in other studies [24]. In terms of results, FFQ showed acceptable reliability among participants.

The correlation coefficient of validity was less than that of reliability. It is possible that some foods were not consumed regularly and were not recorded in the 24-hour dietary recalls. The median intakes for most of the FFQ were higher than the 24-hour dietary recalls. This finding is consistent with other studies [38,39,40] that showed FFQ was overestimated. The study also found that the energy adjustment reduced the correlation coefficients of most nutrients, possibly because the nutrients intakes and types of foods varied from person to person. These results are similar to those of other studies [41,42,43]. The Bland-Altman method was used to evaluate the consistency between the two methods graphically. The results showed that the two methods were comparable, although the average of the differences suggested that FFQ slightly overestimated some nutrients.

It had to be admitted that this study has some limitations. As with all dietary assessment measures, FFQ and 24-hour dietary recall rely on self-reported data, which may be subject to some reporting bias [37]. On the other hand, only two 24-hour dietary recalls were conducted on a random day as there was no significant difference between weekdays and weekends in the diets of residents in areas in this study, and if the number is increased, the consistency between the dietary recall and FFQ will be more authentic and reliable. In addition, this study only used the 24-hour dietary recall for validity study, and the results would be more convincing if biomarkers were used.

Conclusions

In summary, our SFFQ showed acceptable reliability and reasonable validity in assessing food groups and nutrients intakes for the population at high risk for CVD. Based on this study, the SFFQ is suitable for the assessment of dietary intake of people at high risk of CVD in Ningxia, China.