Healthy User Bias Taints Everything

In the early 1950s, prominent University of Minnesota physiologist Ancel Keys, whose work was funded in part by the massive Sugar Research Foundation, began publicly promoting his hypothesis that dietary fat and cholesterol were the main drivers of cardiovascular disease

In 1955, President Eisenhower suffered arguably the most noteworthy heart attack in history, launching heart disease fully into the public consciousness. Taking on Keys as a personal advisor, Eisenhower adopted and began advocating for a “prudent” low-fat diet.

In 1961 the American Heart Association officially condemned saturated fat as a likely cause of heart disease. The AHA was by this time the world’s largest non-profit, largely thanks to significant funding from Proctor & Gamble, makers of the industrial saturated-fat replacement product Crisco. That same year, Ancel Keys appeared on the cover of Time Magazine, depicted as a scientific crusader against the scourge of saturated fat.

1977 saw the initial release of the US government’s first “Dietary Goals for Americans,” which advised Americans, in the interest of health, to strictly limit their consumption of saturated fat, cholesterol, and the animal products that contain them.

By this time, the recommendations were fairly predictable. Keys, Eisenhower, the AHA, publicized congressional hearings, and more contributed to what was fast becoming common knowledge – that the consumption of fatty foods, red meat, saturated fat, and cholesterol were a danger to health and well-being.

In the 1980s, finally, researchers began looking to see whether any of it was actually true.

Epidemiology

Now, I don’t mean to say nutrition research as a whole literally started in the 1980s, only the type you’re most familiar with.^1,2 Nutrition research more broadly started in earnest a couple decades prior when researchers set about trying to legitimize the guidelines that had already been set forth. Often, it should be noted, these studies promptly proved them wrong. For example, major trials such as the Sydney Diet Heart Study and Keys’ own Minnesota Coronary Experiment sharply reduced saturated fat intake for thousands of subjects…who subsequently died at higher rates than the groups who continued consuming their butter and eggs.^3,4 A landmark trial in the 1970s included smoking cessation alongside the “prudent diet” and had only to show for it a slight decrease in cardiovascular disease and a slight increase in overall death.⁵ That ostensibly counterintuitive results like these did nothing to slow the low-fat train speaks to how rapidly and aggressively such advice was being pushed.

Instead, the type of study that had never yet been carried out, but with which you’re most familiar today, is something called nutritional epidemiology. In essence, it is a field that attempts to examine dietary trends at a population level and seek out associations between these trends and various health outcomes. A common approach, for example, is to survey hundreds or thousands of subjects on their dietary habits for the past year (using something called a Food Frequency Questionnaire) and either assess or track changes in various health markers. Researches will ask about other things as well – smoking, exercise, etc. – and attempt to find connections between certain habits and favorable or unfavorable health outcomes.

Epidemiology is a weak science in that it is only an observation, and cannot control many factors in the way a well-designed interventional trial can. For this reason, it can never “prove” anything, although it is a popular brand of nutritional science nonetheless, owing largely to its simplicity and comparatively cheap cost. If you’ve ever seen a news headline linking some food to a given disease, what they were reporting on were the results of nutritional epidemiology.

But there’s one major problem with epidemiology that in many circles never garners the attention it deserves. Because conventional nutrition guidelines and advice predate nutritional epidemiology by decades, there has never been an epidemiological study untainted by the effects of these guidelines. As epidemiological observations increase in rate and conventional guidelines persist year over year, this problem continues to get worse.

Because here’s the thing – it is, for better or worse, common knowledge that meat and saturated fat are “bad” for you. Most adults have heard this for their entire lives, and the majority of them likely believe it to be true. They also believe smoking to be bad for their health, likewise for alcohol, sugar, and a sedentary lifestyle. It should not be surprising, then, that one who partakes in any of these ostensibly damaging behaviors is more likely to engage in additional harmful acts as well. Consider a study linking red meat consumption to diabetes, for example, that found the group eating the most red meat was 40% less active, drank 60% more alcohol, consumed several hundred additional daily calories worth of sugar and refined grain, and were three times more likely to smoke than those who ate the least.⁶ (You can read more over here about how ignoring over 400 calories per day of sugar and refined grains helps create the catchy headline about meat causing diabetes).

 

Insufficient Adjustment

Usually (but not always!), researchers adjust for the most obvious lifestyle habits – exercise, smoking, alcohol – in attempt to prevent them from tainting the results. But what about things like sleep? What about hydration and hygiene and seat belts and safe driving? What about preventative health care? Trying to link a dietary pattern to cancer mortality gets sketchy really quick if one group is more diligent about cancer screenings. The unfortunate reality of epidemiology is that, undoubtedly, the vegetable-eating exercisers who don’t smoke are also going to be healthier in almost every little way you can imagine, even if the researchers don’t adjust for it.

If a study says women who eat the most vegetables are 15% less likely to develop cardiovascular disease, how much of that is actually due to the veggies?⁷ And how much of it is due to the unmeasured practices like those above – sleep habits, preventative health care, risk-taking behavior, and so on? And what if that 15% doesn’t even consider other dietary factors, like sugar? You can be quite certain that those eating the most vegetables were, on average, eating the least sugar. But the researchers in this study didn’t measure or adjust for it, so that’s probably some of our 15% right there.

Unfortunately, its not remotely uncommon for studies like this to completely ignore sugar or any other dietary factor.^8–10 Often, this is even acknowledged once you start reading the actual study. Take for example a meta-analysis (a bunch of studies pooled together for more statistically powerful results) that claims to have found “further evidence that a higher consumption of fruit and vegetables is associated with a lower risk of all-cause mortality” – even while acknowledging that fewer than half of the studies assessed other aspects of the subjects diets.¹¹ They included one that didn’t even adjust for alcohol consumption or exercise!¹² So while they claim that their findings “provide further support for the public health message to increase fruit and vegetable intake”…did they really? Or did they just ask people how many vegetables they eat and found the healthy ones consume more than the sedentary drinkers do.

 

Meat Causes Disease…. Right?

If you’re still looking for more indication that healthy user bias taints epidemiology, consider this study that claims to “provide evidence that dietary modification in choice of protein sources may influence health and longevity.”¹³ They tracked nearly 240,000 (!) men and found that each 3% reduction in red meat consumption was associated with a 7% reduction in cancer mortality, a 12% reduction in cardiovascular death….and an 18% reduction in death due to injury or accident. Another found that men who ate the most red meat (compared to those who ate the least) were at a 22% increased risk of dying from cancer, 27% increased risk of dying from cardiovascular disease, and 26% increased risk of dying in an accident.¹⁴ None of this should be surprising! After all, dangerous drivers are more likely to eat animal based or processed food snacks, while seatbelt use seems to predict chronic disease more strongly than meat consumption does.^15,16 But these behaviors are simply never accounted for in nutritional epidemiology.

The numbers we’re talking about in some of these epidemiological pieces are often tiny, and its impossible to adjust for all the factors that go into a person’s risk for disease – if researchers even try! Remember all the headlines a few years back about bacon causing colorectal cancer, to the tune of an 18% increase if you eat some every day?¹⁷ 18% is a tiny number when considering relative risk – its the difference between a .037% annual risk and a .044% annual risk, a roughly 1 in 14,000 chance your daily bacon will cause colorectal cancer this year.¹⁸ And that’s if we accounted for healthy user bias! Which we certainly did not. Sugar and refined grains are habitually ignored in epidemiology, and you may never find such a study that adjusts for one of the major effects of excess refined carb consumption – elevated insulin levels. Consider that high levels of insulin production may be associated with as much as a 200-300% increase in relative risk of colorectal cancer!^19–21 And we’ve ignored it entirely.

When researchers fail to consider the excess sugar and processed grain consumption of bacon-eaters and the excess insulin that results (among other lifestyle habits left unadjusted), they have fundamentally failed in their task to actually access whether any link exits between bacon and cancer risk. Yet the BBC will still claim unequivocally that bacon causes cancer, further exaggerating the effects of healthy user bias in future epidemiological pieces by pushing the health-conscious to consume even less of the apparently dangerous food item.

 

Conclusion

This entire piece probably makes it sound like I have it out for fruits and veggies while trying hard to defend red meat. But remember, healthy user bias only works in one direction – there are no studies in which vegetables appear unfairly demonized, because its generally healthy people who eat them. Only the foods people have been told are bad (meat, saturated fat, etc.) can be made to look worse than they are, while only foods that people have been told are good (veggies, whole grains, etc.) can be made to look better. And in fact, to some degree, this effect is present in literally every piece of epidemiology you ever read. Regardless of how good or bad each really is, meat and fat will always, without exception, look some degree worse while vegetables and whole grains look better than any of them are in reality.

So, next time a scary headline suggests that steak will literally kill you, remember:

• The study in question did not assess cause and effect, only an association between two items
• The study in question is likely actually measuring an association between red meat + sugar + refined grain consumption and the scary disease, not just red meat and disease
• The researchers did not consider any health behaviors beyond smoking, alcohol, and exercise (hopefully they at least considered those). Sleep, preventative health care, adherence to safety protocol, risk-taking behavior, and other factors were not included in their assessment
• Red meat "causes" more car crashes than it does cases of cancer

 

1.               Mozaffarian D, Rosenberg I, Uauy R. History of modern nutrition science—implications for current research, dietary guidelines, and food policy. BMJ. 2018;361:k2392. doi:10.1136/bmj.k2392

2.               Boeing H. Nutritional epidemiology: New perspectives for understanding the diet-disease relationship? Eur J Clin Nutr. 2013;67(5):424-429. doi:10.1038/ejcn.2013.47

3.               Ramsden CE, Zamora D, Leelarthaepin B, et al. Use of dietary linoleic acid for secondary prevention of coronary heart disease and death: evaluation of recovered data from the Sydney Diet Heart Study and updated meta-analysis. BMJ. 2013;346:e8707. doi:10.1136/bmj.e8707

4.               Ramsden CE, Zamora D, Majchrzak-Hong S, et al. Re-evaluation of the traditional diet-heart hypothesis: analysis of recovered data from Minnesota Coronary Experiment (1968-73). BMJ. 2016;353:i1246. doi:10.1136/bmj.i1246

5.               Multiple Risk Factor Intervention Trial: Risk Factor Changes and Mortality Results. JAMA. 1982;248(12):1465-1477. doi:10.1001/jama.1982.03330120023025

6.               Pan A, Sun Q, Bernstein AM, et al. Red meat consumption and risk of type 2 diabetes: 3 cohorts of US adults and an updated meta-analysis. The American Journal of Clinical Nutrition. 2011;94(4):1088-1096. doi:10.3945/ajcn.111.018978

7.               Liu S, Manson JE, Lee IM, et al. Fruit and vegetable intake and risk of cardiovascular disease: the Women’s Health Study,. The American Journal of Clinical Nutrition. 2000;72(4):922-928. doi:10.1093/ajcn/72.4.922

8.               Bellavia A, Larsson SC, Bottai M, Wolk A, Orsini N. Fruit and vegetable consumption and all-cause mortality: a dose-response analysis1,2,3. The American Journal of Clinical Nutrition. 2013;98(2):454-459. doi:10.3945/ajcn.112.056119

9.               Nakamura K, Nagata C, Oba S, Takatsuka N, Shimizu H. Fruit and Vegetable Intake and Mortality from Cardiovascular Disease Are Inversely Associated in Japanese Women but Not in Men1,2. The Journal of Nutrition. 2008;138(6):1129-1134. doi:10.1093/jn/138.6.1129

10.             Sauvaget C, Nagano J, Hayashi M, Spencer E, Shimizu Y, Allen N. Vegetables and fruit intake and cancer mortality in the Hiroshima/Nagasaki Life Span Study. Br J Cancer. 2003;88(5):689-694. doi:10.1038/sj.bjc.6600775

11.             Wang X, Ouyang Y, Liu J, et al. Fruit and vegetable consumption and mortality from all causes, cardiovascular disease, and cancer: systematic review and dose-response meta-analysis of prospective cohort studies. BMJ. 2014;349(jul29 3):g4490-g4490. doi:10.1136/bmj.g4490

12.             Genkinger JM, Platz EA, Hoffman SC, Comstock GW, Helzlsouer KJ. Fruit, Vegetable, and Antioxidant Intake and All-Cause, Cancer, and Cardiovascular Disease Mortality in a Community-dwelling Population in Washington County, Maryland. American Journal of Epidemiology. 2004;160(12):1223-1233. doi:10.1093/aje/kwh339

13.             Huang J, Liao LM, Weinstein SJ, Sinha R, Graubard BI, Albanes D. Association Between Plant and Animal Protein Intake and Overall and Cause-Specific Mortality. JAMA Internal Medicine. 2020;180(9):1173-1184. doi:10.1001/jamainternmed.2020.2790

14.             Sinha R, Cross AJ, Graubard BI, Leitzmann MF, Schatzkin A. Meat intake and mortality: a prospective study of over half a million people. Arch Intern Med. 2009;169(6):562-571. doi:10.1001/archinternmed.2009.6

15.             Cohen_cornellgrad_0058F_10285.pdf. Accessed June 25, 2023. https://ecommons.cornell.edu/bitstream/handle/1813/51613/Cohen_cornellgrad_0058F_10285.pdf?sequence=1#page=111

16.             Ge Y, He S, Xu Y, Qu W. Effects of dietary patterns on driving behaviours among professional truck drivers: the mediating effect of fatigue. Occup Environ Med. 2021;78(9):669-675. doi:10.1136/oemed-2020-107206

17.             Processed meats do cause cancer - WHO. BBC News. https://www.bbc.com/news/health-34615621. Published October 26, 2015. Accessed June 25, 2023.

18.             USCS Data Visualizations. Accessed June 25, 2023. https://gis.cdc.gov/grasp/USCS/DataViz.html

19.             Kaaks R, Toniolo P, Akhmedkhanov A, et al. Serum C-peptide, insulin-like growth factor (IGF)-I, IGF-binding proteins, and colorectal cancer risk in women. J Natl Cancer Inst. 2000;92(19):1592-1600. doi:10.1093/jnci/92.19.1592

20.             Ma J, Giovannucci E, Pollak M, et al. A Prospective Study of Plasma C-Peptide and Colorectal Cancer Risk in Men. JNCI Journal of the National Cancer Institute. 2004;96(7):546-553. doi:10.1093/jnci/djh082

21.             Tsujimoto T, Kajio H, Sugiyama T. Association between hyperinsulinemia and increased risk of cancer death in nonobese and obese people: A population-based observational study. International Journal of Cancer. 2017;141(1):102-111. doi:10.1002/ijc.30729

The Problematic Paradigm of LDL-C, Part 2

LDL Studies and the Association Between LDL-C and Heart Disease, pt. 1

Previous - Part 1 - The Development of the Lipid-Heart and Diet-Heart Hypotheses

With a background understanding of how and why the lipid-heart and diet-heart hypotheses began to develop, we’ll now look at the research of the time, followed by more recent examples, that both do and do not support the general thesis of the LDL-disease paradigm. As mentioned in the previous installment, one significant study to be published during the rise of Keys, the AHA, and the general cholesterol paradigm was the Framingham study. An ongoing effort that persists to this day, the initial results were reported in 1957 and detailed a general relationship between cholesterol levels and future cardiac events. We’ll start with Framingham before examining several other lines of research concerned with the relationships between saturated fat, cholesterol levels, and heart disease.

Framingham

The Framingham Heart Study is an ongoing cardiovascular disease project that has been tracking, and continues to track, heart disease for the last several decades. The first published results, a 4-year follow up of middle-age Americans, were made available in 1957. Among the published results was a comparison of total cholesterol levels and atherosclerotic heart disease in men over the age of 45. A couple relevant notes – Frist, science in the 1950s tended to use “people” as a synonym for “middle-age white men who smoked a lot.” That’s not meant to discredit the results, just a note of interest. Second, this study predated the technique and practice to specifically measure LDL cholesterol, and instead used the slightly less meaningful total cholesterol. We’ll define and describe all the relevant terms in a coming section, but for now total cholesterol can be thought of as LDL-C plus HDL-C plus a small contribution from triglycerides.

What these results showed was simply that the men with total cholesterol over 260 mg/dl suffered more frequent heart disease during the 4-year follow up than those with lower cholesterol (for reference, standard guidelines today typically recommend a total cholesterol below 200mg/dl). Similar patterns were noted for elevated blood pressure and obesity. While not a specific measure of LDL cholesterol and not as “scientific” as modern research (this was simply a measuring and counting exercise), the pattern was nonetheless unmistakable – those men with elevated total cholesterol were more likely to suffer future heart disease. At a time when the lipid-heart hypothesis was still gaining credence, these results were instrumental in cementing its place in medical and public consciousness.^1,2

Minnesota Coronary Experiment

While the Framingham study was only observational, the Minnesota Coronary Experiment was a well-designed randomized controlled trial, or RCT. This means that subjects were “controlled” for a long period of time in their dietary habits, with results being tracked over time. In this case, the study was conducted on some 10,000 Minnesota mental health patients, whose diet and life were easily controlled, measured, and tracked. Half of the subjects ate food cooked and served with highly saturated fats like butter, while the other half consumed the same food cooked and served with generally unsaturated fats like corn oil and margarine.

What also made the MCE experiment notable was that it the brainchild of Ancel Keys, who designed the study in an effort to validate his beliefs that saturated fat consumption drove cholesterol levels and heart disease. In 1973, after five years of intervention and tracking, data was published demonstrating that the intervention (unsaturated fat) group did indeed have lower LDL-cholesterol levels (LDL-C measures were now common, unlike during the initial Framingham results). While LDL-C levels had fallen only 4 mg/dl in the saturated fat group, they had fallen 32 mg/dl on average in the intervention group. Perhaps Keys was right! This did in fact seem to validate, to at least some degree, the notion that saturated fat consumption influenced cholesterol levels.

You may be already asking the next relevant question – what about disease and death? After all, lower cholesterol is supposed to portend protection from heart disease. Well, that data was simply not published at the time. In fact, it remained unpublished for decades. Only in the last decade, with the death of Keys and his primary co-researcher, were these results made public by his collaborator’s son, who “rescued” the raw data from his father’s personal computer.

The findings may explain why data on death and disease was never published by Keys. That extra 30 mg/dl drop in LDL-C had not in fact saved lives, but was actually associated with a 22% increase in all-cause mortality. Had these results seen the light of day in the early 70s, they may have helped influence a different nutritional landscape in the years that followed. Instead, the data only on LDL lowering, on top of the Framingham results and multifaced efforts to demonize saturated fat, helped end any and all reasonable debate on the nature of heart disease. The fraudulent efforts of Keys, beginning years earlier with a fabricated relationship between fat consumption and disease, continued, as did the now-entrenched notion that saturated fat must be avoided to prevent heart disease.^3,4

Other Early Studies

The MCE was not the only large trial of the time designed to prove the diet-heart and lipid-heart hypotheses and, believe it or not, was not the only one to hide its undesirable results.

The Sydney Diet-Heart Study was another large RCT designed to explore the effects of replacing saturated fats with unsaturated vegetable oils. The results of the SDHS were also slated to be published in 1973. Instead, all that was published was the following:

“It is concluded that because of multiple changes in lifestyle men who have had myocardial infarction are not a good choice for testing the lipid hypothesis” ⁵

After seven years (and who knows how much effort and money) of designing and tracking dietary intervention in a population chosen by the researches, all they decided to publish was an excuse that their chosen population was inappropriate for studying the topic at hand.

Again, the full results eventually became public and, again, they likely betray the reason the researchers chose to hide them in the first place. The SDHS, like the MCE, was not designed to seek objective truth, but to validate a subjective truth that had already been pre-ordained. So when the data showed that the interventional vegetable oil group was 70% more likely to suffer cardiovascular disease and 62% more likely to die, the researches chose to blame their methodology instead.⁶

This is not to say that every study found an increased rate of death and disease in subjects replacing saturated fat. The Oslo Diet Heart Study was perhaps the most prominent study that did not confirm the (hidden) results of the MCE and the SDHS. Instead, 206 heart attack patients consuming a self-selected diet were compared to 206 similar patients consuming a controlled diet rich in unsaturated vegetable oil. The results? In 11 years, the self-selected diet group saw 102 cardiovascular disease deaths and 5 from other disease, while the vegetable oil group saw 88 cardiovascular disease deaths and 12 from other disease, for an approximately 14 percent decrease in cardiovascular mortality and a 7 percent decrease in all disease-related mortality.⁷

With Framingham and Oslo published and promoted and the MCE and SDHS covered up, the inescapable scientific conclusion of the time confirmed what Keys, the AHA, the government, and many doctors of the time already believed – saturated fat must be minimized in order to lower cholesterol levels and prevent death and disease.

** This part is purely my opinion: The Oslo study is absolutely horrible. Why? Because there is no real control group. The MCE and SDHS, for example, controlled the diet of both groups. This means that total calorie consumption, total sugar and carbohydrate consumption, nutrient density, etc. were all roughly equal between the two groups, and the difference in fat could be better assessed as the reason for any differences. Oslo didn’t do that. They simply let the control group eat whatever they wanted and made no effort to track it, while tracking every aspect of the intervention diet. So while the interventional high-vegetable oil group suffered slightly less disease, there isn’t really anything relevant to compare it to. This is a common problem even in modern science, where a population will go from a self-selected diet to one that’s fully controlled. Low-fat, low-carb, vegan, keto, whatever…but the researchers don’t control for calories. So any difference they find after the intervention could, in theory, be attributable to a decrease in calories rather than the dietary pattern the researchers are trying to assess. Because those in the Oslo study went from a self-selected “heart attack diet” to one entirely controlled by the researchers, it is entirely possible that the reduction in disease mortality was due to the intended intervention itself, but… it could also be due a reduction in calories or sugar or something else, and we have no possible way to know that. The increased mortality of the vegetable oil groups in the better-designed MCE and SDHS lend credence to the theory that vegetable oil is not the reason for the decreased mortality in the Oslo study. In fact, it may be the case that the vegetable oil is still causing excess disease, but that this effect is more than counterbalanced by the reduction in disease caused by a decrease in calories, sugar, or any change the researchers aren’t measuring or reporting.

Part 3 - LDL Studies and the Association Between LDL-C and Heart Disease, pt.2

 

 

 

1.           Framingham Heart Study. In: Wikipedia. ; 2022. Accessed December 14, 2022. https://en.wikipedia.org/w/index.php?title=Framingham_Heart_Study&oldid=1126493920
2.           Dawber TR, Moore FE, Mann GV. II. Coronary Heart Disease in the Framingham Study. Am J Public Health Nations Health. 1957;47(4 Pt 2):4-24.
3.           Frantz ID, Dawson EA, Ashman PL, et al. Test of effect of lipid lowering by diet on cardiovascular risk. The Minnesota Coronary Survey. Arteriosclerosis. 1989;9(1):129-135. doi:10.1161/01.atv.9.1.129
4.           Ramsden CE, Zamora D, Majchrzak-Hong S, et al. Re-evaluation of the traditional diet-heart hypothesis: analysis of recovered data from Minnesota Coronary Experiment (1968-73). BMJ. 2016;353:i1246. doi:10.1136/bmj.i1246
5.           Woodhill JM, Palmer AJ, Leelarthaepin B, McGilchrist C, Blacket RB. Low Fat, Low Cholesterol Diet in Secondary Prevention of Coronary Heart Disease. In: Kritchevsky D, Paoletti R, Holmes WL, eds. Drugs, Lipid Metabolism, and Atherosclerosis. Advances in Experimental Medicine and Biology. Springer US; 1978:317-330. doi:10.1007/978-1-4684-0967-3_18
6.           Ramsden CE, Zamora D, Leelarthaepin B, et al. Use of dietary linoleic acid for secondary prevention of coronary heart disease and death: evaluation of recovered data from the Sydney Diet Heart Study and updated meta-analysis. BMJ. 2013;346:e8707. doi:10.1136/bmj.e8707
7.           Leren P. The Oslo Diet-Heart Study. Circulation. 1970;42(5):935-942. doi:10.1161/01.CIR.42.5.935