Whether a particular questionnaire may be considered valid will be largely dependent on the purpose of the study. For example, in case-control studies or intervention trials the mean intakes of groups are compared. The questionnaire must therefore be able to satisfactorily estimate group mean intakes of the nutrients of interest. Studies using a semi-quantitative food frequency questionnaire (e.g. DIETQ) have been reported to yield similar results to those from 7-day weighed intake records. (For nutritional analysis of weighed intake records, see WISP).
In a cross-sectional survey to investigate whether there are associations between diet and disease risk factors such as blood lipids or blood pressure, the dietary survey method chosen needs to be able to rank individuals by their intakes for the foods or nutrients of interest. Recording methods such as the weighed intake record or food diary may be more likely to detect associations than a food intake questionnaire. In the Caerphilly Heart Disease Study, for example, significant associations were detected between dietary variables and plasma lipids, lipoproteins and haemostatic factors when intakes were estimated from 7-day weighed intake records, but many of these associations were much weaker and not statistically significant when intake estimates from semi-quantitative food frequency questionnaire were used. However, recording methods are not always superior: they would not be suitable for items consumed infrequently, e.g. fatty fish. Also, a food frequency questionnaire may be just as likely to detect associations (and be more cost-effective) if the nutrients of interest are present in a few, easily identifiable foods, e.g. vitamin C.
In prospective (cohort) studies, statistical analyses include division of subjects into tertiles or quintiles according to their nutrient intakes. The number of new disease events during the follow-up period is then calculated for each tertile or quintile of the nutrient intake distribution. One can then assess whether those in the top quintile of the intake distribution have a greater or lesser disease risk than those in the bottom quintile and whether there is a trend in disease risk with intake. For this type of study therefore, the questionnaire needs to be able to correctly classify individuals as low, moderate or high consumers of particular foods or nutrients.
The question then is how good does the classification need to be? Typical values for comparison between a semi-quantitative food frequency questionnaire and weighed food intake records are that about 68% of subjects are classified in the same or adjacent quintile and 2% classified in opposite quintiles. In this case, the true difference between the top and bottom quintile is 50% less than the observed difference. This is because although the mean value for the total group is the same, true means for the bottom two quintiles are higher than estimated, and true means for the top two quintiles are lower than estimated, i.e. the slope is less steep as a result of misclassification. This will therefore reduce the ability of the study to detect diet-disease associations. When comparing intake estimates from a food intake questionnaire with those from another (better accepted) method, 80% of subjects would need to be classified in the same fifth of the distribution and none classified in opposite fifths in order to maintain the gradient of the slope from bottom to top quintile of nutrient intake.
Another way of looking at this problem is to examine the effect of misclassification on estimates of relative risk of disease. Comparison of intake estimates from questionnaires with those from weighed intake records or food diaries have reported correlation coefficients of 0.3 - 0.5 for energy, protein, fat and carbohydrate. It has also been shown that if the true relative risk attributable to a dietary variable is 1.5, a correlation coefficient of 0.4 between the estimated intake and true intake will yield a slightly lower estimated relative risk of 1.2. If the true relative risk is 3.0, then the observed relative risk would be 1.6. Higher correlations between the estimated intake and true intake would produce smaller underestimates in the relative risk of disease.
Note:
If using a questionnaire that has been developed and validated by someone else, ensure that you have the computer software package to calculate nutrient intakes from the questionnaire as well as the questionnaire itself. This is because the validity of the questionnaire depends not only on the questions but also how the responses are converted to nutrient intakes. If you do not have the computer software package that converts respones to nutrient intakes, you cannot assume that the validity of the questionnaire will be the same.
