Diet variation, quantification, and misreporting

What is diet variation and why is it important?

A key objective of many studies in terms of dietary assessment is to estimate the habitual dietary intake i.e. the average long-term intake of particular foods or nutrients for the group or individuals under study [41]. Habitual intake is exceedingly difficult to measure due to the diet variation. The diet variation consists of:

1. The differences in dietary intakes between different individuals (between-individual variation)
2. The differences in dietary intakes for the same individual across time (within-individual variation)

Both the between- and within-individual diet variation should be considered during the selection of the method of dietary assessment, as well as in the data analysis stage. Figure D.1.9 shows hypothetical data for daily energy intake for five individuals across four days of measurement. In the top figure (A), between-individual variation is large, as indicated by the wide spread of energy intake across the five individuals (i.e. lines are relatively separated on y-axis). On the other hand, within-individual variation is small, as indicated by stable energy intake across days (i.e. lines are relatively flat).

In the bottom figure (B), between-individual variation is small, as indicated by the lack of the spread of energy intakes across the five individuals (i.e. lines close together on y-axis). In contrast, within-individual variation is large, as shown by the change in energy intake from day-to-day for each individual (i.e. lines are relatively uneven).

For the purpose of describing population prevalence of over- or under-nutrition, understanding of the diet variation is crucial. Bias in the estimation can occur without proper consideration of the within-individual diet variation. For example, if within-individual variation is large and then between-individual variation is over-estimated, the estimate of prevalence of under-nutrition can be biased. Depending on the population average, the estimate of the prevalence can be either under-estimated or over-estimated (Figure D.1.10). The magnitude and direction of bias depends on the nutrient, the method to assess the intake, and the population.

Example using coefficient of variation

Dietary variation can be expressed as the coefficient of variation (CV), a standardised measure of spread that describes the amount of variability relative to the mean. The within-individual and between-individual variation can be expressed using CV as follows:

CV for within-individual variation = [(√within-individual variance) / mean]*100

CV for between-individual variation = [(√between-individual variance) / mean]*100

For most nutrients, within-individual variation is greater than between-individual variation [1 ]. This is demonstrated in Table D.1.10 for a sub group of Japanese women [18 ]. For example, fat intake was on average 59.7g per day, but this varied on average by 35% for each individual from one day to the next, but only by 19.3% from one individual to another. The scenario in which within-individual variation is greater than between-individual variation is shown above in Figure D.1.9 part B.

The relationship between within- and between-individual variation has important implications for assessment, and can be summarised as a ratio (for example see ‘VR’ in Table D.1.10). If within-individual variation is small relative to between-individual variation (see Figure D.1.9 part A), then individuals can be more readily distinguished [1 9]. If, as is normally the case, the within-individual variation is large relative to between-individual variation, then distinguishing between individuals and subsequent ranking is more difficult.

This partly explains why the mean intake of a group is usually more readily assessed than the intake in an individual.

Factors influencing diet variation

Diet variation can be influenced by the following factors:

• Seasonal variation. It is possible that dietary assessment undertaken in the few weeks of a year when a certain fruit is widely available is likely to give a distorted view of habitual micronutrient intake. However, it is possible that with the large amount of imported fruit and vegetables now available in some countries and settings, seasonal variation is of less relevance than in the past when there was much greater reliance on local produce that was seasonal. The costs of foods may also vary with their seasonal availability. This issue should be considered especially relative to the population under study.
• Variation in meal patterns.
• Variation in household composition.
• The wide range of foods available.
• Consumption of food outside the home (e.g. take-aways, restaurants).
• Working patterns.
• Special days including festivities etc.
• Disease states.

Accounting for diet variation

Study sample

Both the within-individual and between-individual diet variation are natural phenomena. Therefore, regardless of a small or large sample size, we cannot reduce the diet variation. However, if we increase the sample size, we can improve estimates (i.e. increase precision) of the within-individual variation, the between-individual variation, or both.

To estimate the within-individual variation and improve the estimates, we need to increase the sample size of repeats from the same participants. To do the same for the between-individual variation, we need to increase the sample size of study participants (i.e. include a greater number of participants). Of note, if we have a huge sample size of study participants, but dietary intakes are assessed only once, we cannot estimate the within-individual variation of the participants.

Selection of method of dietary assessment

• Dietary history: assesses the between-individual variation and the usual intake.
• Food frequency questionnaire (FFQ): assesses the between-individual variation and the usual intake (also depending on the reference period of the questionnaire).
• Weighed or estimated food diaries: assess the between-individual variation and if done on multiple days, they also assess the within-individual variation and it is more probable that they capture the habitual diet.
• 24-hour recalls: assess the between-individual variation and if done on multiple days, they also assess the within-individual variation and it is more probable that they capture the habitual diet.
• Biomarkers of intake: assess the between-individual variation and if there are repeated measures for them, they also assess the within-individual variation.

Time of the year

The greater the seasonal variation, the broader the reference period should be when referring to FFQ; or repeated FFQs should be undertaken to cover usual diet over seasons. When referring to diet records or 24-hour recalls, the greater the seasonal variation, the more widespread the days across the year should be.

If a study plans to assess dietary change over years, investigators may want to collect repeated dietary data at the same time of the year to consistently account for seasonal variation. If the repeated measures were collected in different seasons, changes in dietary intakes would be confounded by the within-individual seasonal variation.

Number of days for multiple dietary records and 24-hour recalls

The factors which should be considered to decide the number of days include the following:

• Within-individual variation [30]: The greater the day-to-day variation in intake, the more days are needed to accurately capture habitual intake. Macronutrients require the shortest number of recording days. The number of days required to accurately reflect micronutrients intake depends on the amounts of the nutrients contained in food e.g. omega 3 fatty acids or vitamin A, and the frequency with which these foods are consumed. The lower the amounts and the frequency, the longer the period of assessment required. The use of a complementary dietary assessment method such as an FFQ may help to better capture infrequently consumed food groups [12]. An increased number of days can affect both recruitment and compliance so a trade-off is required between number of days and participant burden. In the UK, for instance, the current National Diet and Nutrition Survey (NDNS) has been reduced from seven days of assessment to four days (and changed from weighed to estimated records) to improve recruitment and completion.
• The main interest of the study: if the interest is at individual level, more days are needed to account for within-individual variation. If the interest is at group level, fewer days might be needed. If mean intakes of individuals are required, more days of recording are required, e.g. for 90% of the population 13 days of recording are necessary to determine the mean energy intake with a standard error of ± 10% [3]. Further estimates of the number of days required to rank dietary intake are provided by Nelson et al [30].

Day of the week

The biggest difference in dietary intake has often been assumed to be between weekdays and weekends. The assumption often made is that diets on Saturday and Sunday were similar. Analysis from the UK National Diet and Nutrition Surveys (NDNS) of adults and young people has shown that these two days may vary greatly from each other, suggesting that both days may need to be assessed [47, 48].

In studies using interviewer-administered dietary assessment, attention must be paid to the days of the week. Interviewers who do not work weekends or respondents who do not want to be interviewed on weekends for instance may lead to a bias in the dietary data collected.

In estimation of the habitual consumption of dietary intakes from multiple days of dietary records or 24-hour recall, a simple average can cause bias. Suppose a dietary assessment is implemented on one day from the weekend and two days from weekdays. The average of three days over-represents the weekend, because 33% of data come from weekends, rather than 28.6% (2 days out of 7 days). Unless a 7-day assessment is implemented, weighted average should be calculated. In this example, the calculation should be (2 x weekend + 5/2 x (weekday 1 + weekday 2) divided by 7.

In the analysis of 24-hour recall data, days of recording and days of diets should be carefully distinguished. For instance, dietary data collected on Monday represents recalled diet on a Sunday.

Why is portion size important?

The estimation of portion size has been recognised for more than 50 years as a source of error in studies measuring dietary intake [55]. Coefficient of variation of the differences between estimates and weights of food portions have been consistently shown to be around 50% for foods and 20% for nutrients [4 ]. The errors associated with quantifying the portion of food consumed represent probably the largest measurement error in most dietary assessment methods [1 9].

Estimating portion size

Options for estimating portion size include:

Weighing

Portion sizes and leftovers are measured by the participant or by research staff. This increases burden of data collection and is still subject to error.

Household measures

Household measures (e.g. spoon sizes) may be associated with considerable error. The discrepancies in individual ability to estimate food portions with the use of household measures have been shown to be independent of age, body weight, social status and gender but do vary with the food type and true portion size [56].

Food photographs (2 dimensional)

Food photographs [10, 29] are the most commonly used tool to assist the estimation of portion sizes. Showing photos in a computer display is likely to be helpful in an interview-assisted or a computer-based self-administered dietary assessment. However, studies have shown that both adults and children find the estimation of portion sizes using photographs a difficult task, with a tendency to over-report in both populations, but particularly in children [17]. Some foods have been reported to cause greater difficulty than others; for instance, cheese had the largest error range, muesli was consistently over-estimated, and juice incurred the lowest error range in one study [40].

Photographs that depict a portion (amount consumed), or a serving size (amount served), are typically presented as a series of graduated photographs (see Figure D.1.11) for each food item, bound together in an atlas [19].

Developing a food atlas is a time consuming process [31], so where possible an existing tool should be used providing it has been validated in a similar population. Some factors which should be carefully considered are [31]:

1. The size of the image.
2. The number and the range of portion sizes. Multiple photographs seem to result in more accurate estimates.
3. The angle at which the photograph is taken.
4. Whether a colour or black and white photograph is used.
5. Whether other objects are used to give a sense of scale; this may be helpful.
6. The way the food is presented.
7. The population of interest. It is important that the portion sizes in photographs are based on contemporary data in a similar population. One study has shown that children overestimated food weight by 46% on average, using adult food photographs but this was reduced to an over-estimation of 7% when age-appropriate photographs were provided. Adults overestimated portions by 18% using the adult food photographs [15].

A useful description is available [14] of how food photograph portions were developed using data from previous National Diet and Nutrition Surveys (NDNS) in the UK in children and young people. In this study food photographs were compared to a computer tool, the interactive portion size assessment (IPAS), and found to be comparable with good accuracy but poor precision; making it suitable for group estimates not individual estimates.

Food replicas (3 dimensional) and food models (3 dimensional)

Canada is one of the first countries that used the collection of three-dimensional food models in its National Nutrition Survey [19]. Other countries have also included such an aid e.g. United States and New Zealand. The US NHANES (National Health and Nutrition Examination Survey) uses 3-dimensional measurement aids in the dietary component of the survey. The few studies which have compared the use of 2- and 3-dimensional food models suggest that two-dimensional ones are as effective as three [33, 34]. Other studies evaluating food portion aids have shown mixed results and many studies have been carried out in highly controlled and non-representative conditions [40]. Food models may be more reliable than household measures but often only one size is available, introducing bias as individuals report portions which agree with the models available.

Average portion sizes for a given population

Average portion sizes for some foods can be estimated by using existing data [11, 54]. Those data should ideally be kept up to date and be specific to a study population. Standard portion sizes are used in ascribing portions in FFQs. Respondents completing semi-quantitative FFQs may often have difficulty in relating their consumption to pre-defined reference portion sizes [16].

Factors which influence estimation of portion size

The following factors may influence estimates of portion size [41]:

Food characteristics

• Foods of indeterminate shape tend to be associated with larger errors and liquids with smaller errors, otherwise there are no consistent observations
• Larger portion sizes appear more difficult to measure than small portion sizes

Individual characteristics

The following individual characteristics can influence estimation of portion size [29, 41]:

• Perception - the individual’s ability to relate an amount of food which is present in reality to the amount depicted in a photograph
• Conceptualisation - the ability of an individual to make a mental construction of an amount which is not present in reality and to relate that to a photograph
• Memory - which affects the precision of the conceptualisation process
• Women have on occasion been shown to provide better estimates than men; this is likely due to their greater role in food preparation
• Training of the respondents might reduce the error incurred

What is misreporting?

Misreporting is a form of respondent bias documented in studies using subjective methods of dietary assessment, and includes both under- and over-reporting.

The detection of misreporting is an investigation of the validity of the dietary data obtained. Often energy intake is used as a proxy for dietary intake in investigations, because if energy intake is underestimated, it is probable that the intakes of other nutrients are also underestimated [23]. Validation of dietary assessment methods therefore often concentrates on estimates of energy expenditure. The development of doubly labelled water (DLW) as a gold standard measure of energy expenditure was thus fundamental to advancing this area of dietary assessment and its links with health outcomes like obesity [43].

Misreporting in dietary assessment was extensively studied during the late 1980s / early 1990s. At that time, the focus of obesity research was on identifying a metabolic cause of obesity, as self-reported dietary intakes did not show intakes that were higher than in lean counterparts [35].

In recent decades, diet quality, in addition to quantity, has received much attention. To study diet quality independent of diet quantity in population-health research, overall dietary habits and dietary intakes have been assessed after adjustment for total energy intake. Therefore, although literature on misreporting largely focuses on validity of total energy intake, it is crucial to consider validity of other dietary intakes after adjustment for total energy intake.

What is the profile of misreporters?

Characteristics of misreporters are population-specific and have not been consistent in the published literature [36, 49]. It has been demonstrated that individuals who misreport on one occasion are likely to do so on subsequent occasions, therefore repeated or more prolonged periods of assessment may not overcome this source of error [8]. Characteristics that influence degrees of misreporting have been studied and include some features as summarised below [21, 22, 27, 28, 32, 37,38, 46, 50]:

• Perception of social desirability plays an important role in misreporting.
• Body size measured by the body mass index (BMI) seems to be the most consistent determinant of energy misreporting in subjective methods of dietary assessment such as FFQ and 24-hour recall. One study showed that an underestimation of energy and also protein and potassium also happens in the duplicate diets method, which indicates a behavioural change during the period of dietary assessment (reactivity bias). Overweight and obese individuals underreport more than lean counterparts.
• Other potential determinants of misreporting that have been reported in the literature but show lower consistency include age, gender, education level, smoking, dietary behaviours such as restrictive eating, emotional eating, eating outside of home and knowledge on healthy eating, BMI-related factors such as body image, dieting, weight difference in the past years, physical activity and body fat and chronic disease status.

Implications of misreporting

Misreporting can potentially lead to the following:

• Attenuation of any diet and disease relationships, particularly in the study of obesity [23].
• Limited and insufficient definition of food patterns and the subsequent derivation of food based guidelines [2].
• Estimates of different food intakes are affected differentially, with healthy foods often overreported and unhealthy foods often underreported [22].

Accounting for misreporting

Doing nothing may be considered an option, with caveats

If the research aim is to rank individuals and the ability of ranking individuals is well supported objectively (e.g. in a validation study), systematic misreporting can be ignored in data analysis. For example, if 100% of study participants underestimate fruits and vegetable consumption by 10%, ranking of individuals is not influenced. Similarly the association of diet intakes with a health outcome can be unchanged even in presence of over- and under-reporters (refer to systematic errors page). Therefore, the need and approach of accounting for misreporting should be considered with a research aim.

It is practically difficult to ensure that systematic errors are not differential. Although doing nothing is an option, analysis after accounting for possible over- and under-reporting of total energy intake is generally recommended in dietary research.

Estimation of energy intake with reference to estimated total energy expenditure

As noted above, a focus on misreporting tends to concentrate on estimation of absolute levels of total energy intake. Habitual total energy intake should match with habitual total energy expenditure under assumption that participant weights are static (in a state of energy balance). Under this assumption, estimates of total energy expenditure are often used to identify mis-reporters in dietary research.

Ideally, studies would measure total energy expenditure (TEE) by doubly labelled water (DLW), but this is expensive (approx. £1000 per dose). Therefore, a typical population-health study estimates TEE by basic characteristics that are readily measured. The estimation uses standard equations of basal metabolic rate (BMR) [44], plus energy expenditure due to physical activity level (PAL). Estimates of TEE are improved if physical activity is assessed objectively. A direct measure of BMR can be made by calorimetry.

TEE comprises energy expenditure due to physical activity and also digestion of meals. The ‘thermic effect of food’ can be estimated from dietary consumption, but is typically ignored because TEE, used as a reference to subjective total energy intake, should be independent of dietary measures assessed simultaneously. Instead, PAL is assigned to be a constant coefficient that captures energy expenditure in addition to BMR. The following summary of PAL values has been calculated from DLW measurements of TEE. For each activity, it is assumed that overall activity is as detailed in table D.1.11.

In the literature, use of TEE estimates is inconsistent and a topic of methodological research [25, 26, 39, 49, 51]. The following are examples of methods to distinguish individuals who are under-reporters, plausible reporters and over-reporters or to control for magnitude of over- and under-reporting.

I. The ratio of total energy intake to TEE

Different approaches of using the ratio can be undertaken. A study may exclude individuals based on certain cut-points defined within that study, for example bottom and/or top percentiles and means ±1 standard deviation (Note the standard deviation represents within-individual variation or uncertainty of the ratio). Some more recent studies have suggested statistical adjustment for the ratio to hold degrees of misreporting consistent in a population and stratification of results based on misreporting status to avoid unnecessary loss of power and introduction of unpredictable selection bias [26, 49, 51].

Using the ratio of total energy intake to TEE, when examining relationships between diet and health in a US national cohort, the use of ±1 SD cut-offs was reported to be preferable to the ±2 SD cut-offs for excluding inaccurate reports [25]. Mendez et al [26] reported that the use of such alternative methods had stronger influence on associations compared to using the Goldberg method for instance in the Spanish EPIC cohort.

II. Goldberg cut-offs

Goldberg cut-offs have been used in dietary surveys or studies that aim to assess absolute levels of total energy intake and other dietary components. Goldberg and colleagues [20] developed two cut-offs for the agreement between PAL and the ratio of reported energy intake to BMR. The application was first demonstrated by Black et al [9]. In the first, ‘CUT-OFF 1’, PAL was set at 1.35, the minimum plausible value for most individuals who are weight stable. Subsequent work in this area led to a recommendation that this cut-off should no longer be used because it fails to account for biological variability and measurement error in estimating both energy intake and expenditure [6]. It was also noted that it underestimated underreporting in individuals whose activity was above a sedentary level.

‘CUT-OFF 2’ differs from the previous cut-off as total energy expenditure or PAL varies according to the population or individuals under study. As with CUT-OFF 1 it involves a statistical comparison between reported energy intake and BMR accounting for biological variability and measurement error. Originally only a lower cut-off was developed but if the activity is known or can be assumed, an upper limit can be determined [6] i.e. the 95% confidence limits for the ratio of reported energy intake to BMR and PAL.

Researchers should state the criteria used for calculating the cut-offs, and include such information in any publications, as there is potential for inappropriate use of the cut-offs, through a lack of thorough understanding of the principles [23, 25].

III. Adjustment for predictors of energy balance such as BMI

More recent studies have suggested that use of the ratio of EI to TEE in analyses where the outcome is correlated with TEE such as anthropometric outcomes (e.g. fat mass, fat-free mass) may result in selection bias. An alternative proposed is the adjustment for predictors of energy balance, which has been shown to result in associations with energy intake closer to those of a method of objective measure of energy intake [50].

For misreporting of dietary intakes, rather than total energy intake, bias due to misreporting could be reduced in a population through adjustment for total energy intake [52]. This relates to the topic of errors in multiple variables (e.g. food consumption and energy intake).

IV. Use of predefined lower and upper limits of energy intake

To estimate TEE without an objective measurement (e.g. DLW), equations involve the use of information on weight and height. If the weight is not objectively measured, weight may be over- or under-estimated and TEE estimation can be biased and not useful anymore. To avoid this issue, a recent study proposed predefined population-specific limits of energy intake to identify mis-reporters, such as energy intake <500kcal/day and >3,500 kcal/day for women [39].

Recommendations

• Conduct or refer to a validation study in which systematic and random errors should be observed to be minimal after accounting for possible mis-reporters.
• Cut-offs can provide help for identifying underreporters.
• Measure physical activity, ideally by an objective method such as accelerometry, to improve estimates of energy expenditure. Alternatively, subjective methods such as a physical activity questionnaire can be used to rank individuals according to activity level but have limited validity for estimating the absolute energy expenditure of individuals.
• Assess dietary restraint if possible.
• Any researcher working with study individuals must be aware that their attitude, general demeanour, and non-verbal and verbal language convey messages which may affect the likelihood of underreporting, such as through social desirability bias.
• Conducting focus groups to explore behaviours around dietary assessment may provide an insight into the likelihood of misreporting [8].
• Conduct additional analyses taking different approaches of accounting for misreporting (i.e. sensitivity analysis) to assess stability of results.