Did you know that being healthy can actually skew the results of scientific studies? It’s called “healthy user bias,” and it plays a major role in how we interpret nutrition research. In this video, we explore how healthy user bias impacts the validity of studies claiming that eating more plant protein and less animal protein reduces heart disease risk. You’ll learn why these claims often fall apart under scrutiny and how to think critically about such headlines.
Transcript:
Plant vs. Animal Protein: How Healthy User Bias Misleads Nutrition Science
Did you know that being healthy can ruin a scientific study? It’s called healthy user bias. You may have heard of it before, but you may not realize how it can ruin a study’s conclusions. So let’s talk about healthy user bias in the context of recent headlines that eating more plant protein unless animal protein reduces your risk of heart disease. And we’ll see how that evidence kind of falls apart when we consider healthy or unhealthy user bias. Welcome to Metabolic Mind, a nonprofit initiative of Baszucki Group, transforming the study and treatment of mental disorders by exploring the connection between metabolism and brain health. Thank you for joining us on this journey.
Study Conclusions
As I mentioned, the study headlines are clear. The recent study concluded eating more plant protein and less animal protein reduces your risk of heart disease. Now, that sounds compelling, right? And it fits the common narrative. So many experts don’t feel the need to dig any further, but if we do dig into the details, we see that the data fall apart and really have no meaningful impact on our food or health choices.
Weaknesses of Nutrition Epidemiology
Now before we go further though, please remember our channels for our informational purposes only, we’re not providing individual or group medical or healthcare advice or establishing a provider patient relationship. Many of the things that we discuss can be dangerous if done without supervision. So please always consult your physician, your healthcare team before changing your lifestyle or medications. So first off, I’m mostly talking about nutrition epidemiology studies. Studies where researchers will follow thousands of individuals for years or even decades, and they have them fill out a couple questionnaires over that timeframe about what they ate, and then they crunch all the data to see who has heart attacks, who doesn’t, who gets diabetes, who dies early. And if what they ate may have contributed to their health issues. As we’ve discussed in the past, these studies have numerous weaknesses including, you know, the infrequent subjective analysis of what people eat leading to pretty low quality data collection. They often have very small hazard ratios, meaning the overall effect is small, so it’s hard to know what the findings mean for you as an individual. And that’s just to name a few.
Healthy User Bias & Impact on Study Results
But perhaps the most concerning is the concept of healthy user bias or unhealthy user bias, meaning one group just happens to be much healthier than the other, and it has nothing to do with the specific variable being measured. I. Like what kind of protein someone ate. And here’s the thing, this is super easy to detect in this example. All you have to do is go to the results section in the baseline characteristics and compare the highest to the lowest quartile of plant versus animal protein ratio. And you see a significant difference. Those that ate more animal protein we’re more likely to smoke, to drink alcohol, less likely to exercise. More likely to have diabetes or hypertension. So when you see this difference, it’s actually no surprise that there’s an increased risk of heart disease in the group that has diabetes and high blood pressure, and smokes and drinks, and doesn’t exercise, right? Nobody’s surprised by that. It’s actually more surprising that there wasn’t a difference in the risk for stroke, but we’ll talk about more of that later. Now it’s important to note that researchers can try to adjust for the smokers or those with pre-existing diabetes or hypertension, et cetera, but we have to acknowledge that this is statistical guesswork and not an exact science. So we can’t say for sure that these adjustments get it right, but we also have to acknowledge that, ah, it’s foolish to believe that we can correct for every aspect of someone’s health. For instance, looking at the baseline characteristics, who’s more likely to get eight hours of sleep? I. Or, you know, do meditation and manage their stress or have more meaningful social interactions or a more meaningful job, a sense of purpose, right? Or who’s more likely to see their doctor for preventive care or be more proactive in their health. Now, of course, it’s not a hundred percent across the board, but for the majority, it’s not likely to be the people who smoke and drink alcohol and don’t exercise. Here’s the thing. The study can’t adjust for all these factors, and you better believe that they can impact the results of a study. So when you see results like this, those who ate more plant protein, less animal protein, had quote unquote better health, you have to ask how confident am I that it was the difference in protein intake that led to the difference in outcomes and that it wasn’t influenced by the baseline level of health between the subjects. Unfortunately, we have to conclude that we really can’t be confident at all in that conclusion. And thus, the grand headlines about how this study helps us understand much more about animal plant proteins is simply wrong.
Applying Study Results to Real Life
You see, one of the main reasons to consider healthy user or unhealthy user bias is not only to understand the validity of the statistics, but to ask. Does this data reflect me or the patient I’m seeing right now? And that’s where this study kind of falls apart in providing general advice. For example, the lowest carb consumption group was at around 40% of their daily calories. So would this data apply to someone using ketogenic therapy to treat a metabolic or brain-based disorder? No, it wouldn’t. Not at all. And I don’t wanna imply that this is a unique issue with this study.
Healthy User Bias in Other Low-Carb/Keto Studies
For comparison, here’s a nutrition epidemiology study that concluded those eating low carb diets were at higher risk of premature death or heart related death. Again, we can see the lowest group was eating 214 grams of carbs per day. That was the lowest, nowhere near a true low carb diet, and people in that group were more likely to be men, to have less than a high school education, be more likely to smoke and drink alcohol, and more likely to have diabetes, clearly unhealthy user bias and just for good measure. Here’s one more study concluding that red meat increases the risk for heart disease comparing the highest to lowest quintile of red meat consumption. Those in the highest group were more likely to be smokers, get less physical activity. And in this study they ate almost a thousand calories more per day. 1000 calories. Talk about a bias, right? So you’re seeing a pattern because I sure am. And when the effect sizes are so small with hazard ratios of 1.1, 1.2, 1.4, there’s no way we can tell that it’s a true effect instead of an unhealthy user bias effect. So does healthy user bias render a study useless? Well, a number of opponents may say yes, and I think there’s no question that it diminishes the value of the findings significantly. But these studies can raise questions that should then go on to be studied with better, more reliable study designs where we can now, we have to acknowledge it’s hard to do the randomized control trial in 10,000 people for 20 years. So that’s where these studies fill a role, but that doesn’t make them high quality evidence. But here’s an interesting twist. Studies with healthy or unhealthy user bias can actually be really instructive in the negative. And here’s what I mean. If a population is less healthy at baseline, but does not have a worse outcome. And that’s a pretty good marker that whatever they were doing wasn’t all that harmful, at least for the time frame studied. Some may even say it was protective, although the strength of that evidence is also low. So in this case, the less healthy population at baseline did not have an increased risk of stroke despite eating a higher percentage of animal foods. So, as I said, the data quality’s still low, but how do we explain that animal foods are so obviously bad for vascular health. Well, I’ll just leave that question there because there isn’t a good explanation, but likely isn’t all that harmful. Right. Okay.
Media Response to Study & Our Takeaways
Now, despite the concerns with unhealthy user bias, I predict we’re going to see the standard response to the study that it adds to the science suggesting we would all be healthier if we switch to more plant protein. But unfortunately. Just about every single one of those studies fails the health of the user bias test, and therefore, the science does not adequately answer the question if the health outcome difference is due to the type of protein eaten or simply due to the baseline health characteristics of the individuals. And if we’re going to use data to make sweeping recommendations for what an entire population should eat, I would hope that we would do that based on very strong data, with very little chance of us being misled. Unfortunately, that’s not the case here. So when you see data like this, remember to ask yourself, could there be an unhealthy user bias influencing the results? If the answer is yes, realize that the data is weaker than it appears from a statistical standpoint and probably doesn’t have the impact that many think it might. I hope this discussion was helpful. You know, to give you a different perspective on nutrition science. If it was, please like and subscribe so you won’t miss any of our future content. And please share this video with those who it may help. And as always, please leave us a comment. I’d love to hear your thoughts. Did I miss something or overstate something, or what do you think about the study in question? So thank you for watching. I’m Dr. Bret Scher, medical Director at Baszucki Group, and we’ll see you here next time at Metabolic Mind, a nonprofit initiative of Baszucki Group.
