Healthy User Bias Taints Everything

In the early 1950s, prominent University of Minnesota physiologist Ancel Keys, whose work was funded in part by the massive Sugar Research Foundation, began publicly promoting his hypothesis that dietary fat and cholesterol were the main drivers of cardiovascular disease

In 1955, President Eisenhower suffered arguably the most noteworthy heart attack in history, launching heart disease fully into the public consciousness. Taking on Keys as a personal advisor, Eisenhower adopted and began advocating for a “prudent” low-fat diet.

In 1961 the American Heart Association officially condemned saturated fat as a likely cause of heart disease. The AHA was by this time the world’s largest non-profit, largely thanks to significant funding from Proctor & Gamble, makers of the industrial saturated-fat replacement product Crisco. That same year, Ancel Keys appeared on the cover of Time Magazine, depicted as a scientific crusader against the scourge of saturated fat.

1977 saw the initial release of the US government’s first “Dietary Goals for Americans,” which advised Americans, in the interest of health, to strictly limit their consumption of saturated fat, cholesterol, and the animal products that contain them.

By this time, the recommendations were fairly predictable. Keys, Eisenhower, the AHA, publicized congressional hearings, and more contributed to what was fast becoming common knowledge – that the consumption of fatty foods, red meat, saturated fat, and cholesterol were a danger to health and well-being.

In the 1980s, finally, researchers began looking to see whether any of it was actually true.

Epidemiology

Now, I don’t mean to say nutrition research as a whole literally started in the 1980s, only the type you’re most familiar with.^1,2 Nutrition research more broadly started in earnest a couple decades prior when researchers set about trying to legitimize the guidelines that had already been set forth. Often, it should be noted, these studies promptly proved them wrong. For example, major trials such as the Sydney Diet Heart Study and Keys’ own Minnesota Coronary Experiment sharply reduced saturated fat intake for thousands of subjects…who subsequently died at higher rates than the groups who continued consuming their butter and eggs.^3,4 A landmark trial in the 1970s included smoking cessation alongside the “prudent diet” and had only to show for it a slight decrease in cardiovascular disease and a slight increase in overall death.⁵ That ostensibly counterintuitive results like these did nothing to slow the low-fat train speaks to how rapidly and aggressively such advice was being pushed.

Instead, the type of study that had never yet been carried out, but with which you’re most familiar today, is something called nutritional epidemiology. In essence, it is a field that attempts to examine dietary trends at a population level and seek out associations between these trends and various health outcomes. A common approach, for example, is to survey hundreds or thousands of subjects on their dietary habits for the past year (using something called a Food Frequency Questionnaire) and either assess or track changes in various health markers. Researches will ask about other things as well – smoking, exercise, etc. – and attempt to find connections between certain habits and favorable or unfavorable health outcomes.

Epidemiology is a weak science in that it is only an observation, and cannot control many factors in the way a well-designed interventional trial can. For this reason, it can never “prove” anything, although it is a popular brand of nutritional science nonetheless, owing largely to its simplicity and comparatively cheap cost. If you’ve ever seen a news headline linking some food to a given disease, what they were reporting on were the results of nutritional epidemiology.

But there’s one major problem with epidemiology that in many circles never garners the attention it deserves. Because conventional nutrition guidelines and advice predate nutritional epidemiology by decades, there has never been an epidemiological study untainted by the effects of these guidelines. As epidemiological observations increase in rate and conventional guidelines persist year over year, this problem continues to get worse.

Because here’s the thing – it is, for better or worse, common knowledge that meat and saturated fat are “bad” for you. Most adults have heard this for their entire lives, and the majority of them likely believe it to be true. They also believe smoking to be bad for their health, likewise for alcohol, sugar, and a sedentary lifestyle. It should not be surprising, then, that one who partakes in any of these ostensibly damaging behaviors is more likely to engage in additional harmful acts as well. Consider a study linking red meat consumption to diabetes, for example, that found the group eating the most red meat was 40% less active, drank 60% more alcohol, consumed several hundred additional daily calories worth of sugar and refined grain, and were three times more likely to smoke than those who ate the least.⁶ (You can read more over here about how ignoring over 400 calories per day of sugar and refined grains helps create the catchy headline about meat causing diabetes).

 

Insufficient Adjustment

Usually (but not always!), researchers adjust for the most obvious lifestyle habits – exercise, smoking, alcohol – in attempt to prevent them from tainting the results. But what about things like sleep? What about hydration and hygiene and seat belts and safe driving? What about preventative health care? Trying to link a dietary pattern to cancer mortality gets sketchy really quick if one group is more diligent about cancer screenings. The unfortunate reality of epidemiology is that, undoubtedly, the vegetable-eating exercisers who don’t smoke are also going to be healthier in almost every little way you can imagine, even if the researchers don’t adjust for it.

If a study says women who eat the most vegetables are 15% less likely to develop cardiovascular disease, how much of that is actually due to the veggies?⁷ And how much of it is due to the unmeasured practices like those above – sleep habits, preventative health care, risk-taking behavior, and so on? And what if that 15% doesn’t even consider other dietary factors, like sugar? You can be quite certain that those eating the most vegetables were, on average, eating the least sugar. But the researchers in this study didn’t measure or adjust for it, so that’s probably some of our 15% right there.

Unfortunately, its not remotely uncommon for studies like this to completely ignore sugar or any other dietary factor.^8–10 Often, this is even acknowledged once you start reading the actual study. Take for example a meta-analysis (a bunch of studies pooled together for more statistically powerful results) that claims to have found “further evidence that a higher consumption of fruit and vegetables is associated with a lower risk of all-cause mortality” – even while acknowledging that fewer than half of the studies assessed other aspects of the subjects diets.¹¹ They included one that didn’t even adjust for alcohol consumption or exercise!¹² So while they claim that their findings “provide further support for the public health message to increase fruit and vegetable intake”…did they really? Or did they just ask people how many vegetables they eat and found the healthy ones consume more than the sedentary drinkers do.

 

Meat Causes Disease…. Right?

If you’re still looking for more indication that healthy user bias taints epidemiology, consider this study that claims to “provide evidence that dietary modification in choice of protein sources may influence health and longevity.”¹³ They tracked nearly 240,000 (!) men and found that each 3% reduction in red meat consumption was associated with a 7% reduction in cancer mortality, a 12% reduction in cardiovascular death….and an 18% reduction in death due to injury or accident. Another found that men who ate the most red meat (compared to those who ate the least) were at a 22% increased risk of dying from cancer, 27% increased risk of dying from cardiovascular disease, and 26% increased risk of dying in an accident.¹⁴ None of this should be surprising! After all, dangerous drivers are more likely to eat animal based or processed food snacks, while seatbelt use seems to predict chronic disease more strongly than meat consumption does.^15,16 But these behaviors are simply never accounted for in nutritional epidemiology.

The numbers we’re talking about in some of these epidemiological pieces are often tiny, and its impossible to adjust for all the factors that go into a person’s risk for disease – if researchers even try! Remember all the headlines a few years back about bacon causing colorectal cancer, to the tune of an 18% increase if you eat some every day?¹⁷ 18% is a tiny number when considering relative risk – its the difference between a .037% annual risk and a .044% annual risk, a roughly 1 in 14,000 chance your daily bacon will cause colorectal cancer this year.¹⁸ And that’s if we accounted for healthy user bias! Which we certainly did not. Sugar and refined grains are habitually ignored in epidemiology, and you may never find such a study that adjusts for one of the major effects of excess refined carb consumption – elevated insulin levels. Consider that high levels of insulin production may be associated with as much as a 200-300% increase in relative risk of colorectal cancer!^19–21 And we’ve ignored it entirely.

When researchers fail to consider the excess sugar and processed grain consumption of bacon-eaters and the excess insulin that results (among other lifestyle habits left unadjusted), they have fundamentally failed in their task to actually access whether any link exits between bacon and cancer risk. Yet the BBC will still claim unequivocally that bacon causes cancer, further exaggerating the effects of healthy user bias in future epidemiological pieces by pushing the health-conscious to consume even less of the apparently dangerous food item.

 

Conclusion

This entire piece probably makes it sound like I have it out for fruits and veggies while trying hard to defend red meat. But remember, healthy user bias only works in one direction – there are no studies in which vegetables appear unfairly demonized, because its generally healthy people who eat them. Only the foods people have been told are bad (meat, saturated fat, etc.) can be made to look worse than they are, while only foods that people have been told are good (veggies, whole grains, etc.) can be made to look better. And in fact, to some degree, this effect is present in literally every piece of epidemiology you ever read. Regardless of how good or bad each really is, meat and fat will always, without exception, look some degree worse while vegetables and whole grains look better than any of them are in reality.

So, next time a scary headline suggests that steak will literally kill you, remember:

• The study in question did not assess cause and effect, only an association between two items
• The study in question is likely actually measuring an association between red meat + sugar + refined grain consumption and the scary disease, not just red meat and disease
• The researchers did not consider any health behaviors beyond smoking, alcohol, and exercise (hopefully they at least considered those). Sleep, preventative health care, adherence to safety protocol, risk-taking behavior, and other factors were not included in their assessment
• Red meat "causes" more car crashes than it does cases of cancer

 

1.               Mozaffarian D, Rosenberg I, Uauy R. History of modern nutrition science—implications for current research, dietary guidelines, and food policy. BMJ. 2018;361:k2392. doi:10.1136/bmj.k2392

2.               Boeing H. Nutritional epidemiology: New perspectives for understanding the diet-disease relationship? Eur J Clin Nutr. 2013;67(5):424-429. doi:10.1038/ejcn.2013.47

3.               Ramsden CE, Zamora D, Leelarthaepin B, et al. Use of dietary linoleic acid for secondary prevention of coronary heart disease and death: evaluation of recovered data from the Sydney Diet Heart Study and updated meta-analysis. BMJ. 2013;346:e8707. doi:10.1136/bmj.e8707

4.               Ramsden CE, Zamora D, Majchrzak-Hong S, et al. Re-evaluation of the traditional diet-heart hypothesis: analysis of recovered data from Minnesota Coronary Experiment (1968-73). BMJ. 2016;353:i1246. doi:10.1136/bmj.i1246

5.               Multiple Risk Factor Intervention Trial: Risk Factor Changes and Mortality Results. JAMA. 1982;248(12):1465-1477. doi:10.1001/jama.1982.03330120023025

6.               Pan A, Sun Q, Bernstein AM, et al. Red meat consumption and risk of type 2 diabetes: 3 cohorts of US adults and an updated meta-analysis. The American Journal of Clinical Nutrition. 2011;94(4):1088-1096. doi:10.3945/ajcn.111.018978

7.               Liu S, Manson JE, Lee IM, et al. Fruit and vegetable intake and risk of cardiovascular disease: the Women’s Health Study,. The American Journal of Clinical Nutrition. 2000;72(4):922-928. doi:10.1093/ajcn/72.4.922

8.               Bellavia A, Larsson SC, Bottai M, Wolk A, Orsini N. Fruit and vegetable consumption and all-cause mortality: a dose-response analysis1,2,3. The American Journal of Clinical Nutrition. 2013;98(2):454-459. doi:10.3945/ajcn.112.056119

9.               Nakamura K, Nagata C, Oba S, Takatsuka N, Shimizu H. Fruit and Vegetable Intake and Mortality from Cardiovascular Disease Are Inversely Associated in Japanese Women but Not in Men1,2. The Journal of Nutrition. 2008;138(6):1129-1134. doi:10.1093/jn/138.6.1129

10.             Sauvaget C, Nagano J, Hayashi M, Spencer E, Shimizu Y, Allen N. Vegetables and fruit intake and cancer mortality in the Hiroshima/Nagasaki Life Span Study. Br J Cancer. 2003;88(5):689-694. doi:10.1038/sj.bjc.6600775

11.             Wang X, Ouyang Y, Liu J, et al. Fruit and vegetable consumption and mortality from all causes, cardiovascular disease, and cancer: systematic review and dose-response meta-analysis of prospective cohort studies. BMJ. 2014;349(jul29 3):g4490-g4490. doi:10.1136/bmj.g4490

12.             Genkinger JM, Platz EA, Hoffman SC, Comstock GW, Helzlsouer KJ. Fruit, Vegetable, and Antioxidant Intake and All-Cause, Cancer, and Cardiovascular Disease Mortality in a Community-dwelling Population in Washington County, Maryland. American Journal of Epidemiology. 2004;160(12):1223-1233. doi:10.1093/aje/kwh339

13.             Huang J, Liao LM, Weinstein SJ, Sinha R, Graubard BI, Albanes D. Association Between Plant and Animal Protein Intake and Overall and Cause-Specific Mortality. JAMA Internal Medicine. 2020;180(9):1173-1184. doi:10.1001/jamainternmed.2020.2790

14.             Sinha R, Cross AJ, Graubard BI, Leitzmann MF, Schatzkin A. Meat intake and mortality: a prospective study of over half a million people. Arch Intern Med. 2009;169(6):562-571. doi:10.1001/archinternmed.2009.6

15.             Cohen_cornellgrad_0058F_10285.pdf. Accessed June 25, 2023. https://ecommons.cornell.edu/bitstream/handle/1813/51613/Cohen_cornellgrad_0058F_10285.pdf?sequence=1#page=111

16.             Ge Y, He S, Xu Y, Qu W. Effects of dietary patterns on driving behaviours among professional truck drivers: the mediating effect of fatigue. Occup Environ Med. 2021;78(9):669-675. doi:10.1136/oemed-2020-107206

17.             Processed meats do cause cancer - WHO. BBC News. https://www.bbc.com/news/health-34615621. Published October 26, 2015. Accessed June 25, 2023.

18.             USCS Data Visualizations. Accessed June 25, 2023. https://gis.cdc.gov/grasp/USCS/DataViz.html

19.             Kaaks R, Toniolo P, Akhmedkhanov A, et al. Serum C-peptide, insulin-like growth factor (IGF)-I, IGF-binding proteins, and colorectal cancer risk in women. J Natl Cancer Inst. 2000;92(19):1592-1600. doi:10.1093/jnci/92.19.1592

20.             Ma J, Giovannucci E, Pollak M, et al. A Prospective Study of Plasma C-Peptide and Colorectal Cancer Risk in Men. JNCI Journal of the National Cancer Institute. 2004;96(7):546-553. doi:10.1093/jnci/djh082

21.             Tsujimoto T, Kajio H, Sugiyama T. Association between hyperinsulinemia and increased risk of cancer death in nonobese and obese people: A population-based observational study. International Journal of Cancer. 2017;141(1):102-111. doi:10.1002/ijc.30729