Failing GRADE for Annals of Internal Medicine Meat Studies

Below is an approximation of this video’s audio content. To see any graphs, charts, graphics, images, and quotes to which Dr. Greger may be referring, watch the above video.

A series of articles published in the Annals of Internal Medicine culminated in a recommendation suggesting people keep eating their red and processed meat. Nutrition researchers savaged these articles. The chair of the nutrition department at Harvard called it “a very irresponsible public health recommendation,” and the past Harvard nutrition chair was even less restrained. “It’s the most egregious abuse of data I’ve ever seen,” said Walter Willett, “There are just layers and layers of problems.” Let us start to pick through these layers.

The first of several serious weaknesses was that the analyses and recommendations were largely based on the so-called GRADE criteria (Grading of Recommendations, Assessment, Development, and Evaluation), which I talked about in my last video. The authors erred in applying the GRADE tool, since that was designed for drug trials.

GRADE automatically scores observational studies as “low- or very-low” scores for “certainty of evidence,” which is exactly what you want when you’re evaluating evidence from drug trials. You want a randomized double-blind, placebo-controlled trial to prove the drug’s risks and benefits. However, the infeasibility for conducting randomized clinical trials on most dietary, lifestyle, and environmental exposures makes the criteria inappropriate in these areas, since it would involve controlling people’s daily diet and following them for decades.

“You can’t do a double-blinded placebo-controlled trial of red meat and other foods on heart attacks or cancer. “For dietary and lifestyle factors, it’s impossible to use the same standards for drug trials.” Imagine telling one group of people to smoke a pack of cigarettes every day for the next 20 years to prove that cigarettes cause lung cancer. And how could you make it double-blind—have the control group smoke placebo cigarettes?

Yet, in the meat papers, they were downgrading studies due to lack of blinding. Well duh, in nutritional trials how are you going to blind people to the facts of what they’re eating? GRADE is just the wrong tool for diet studies. In fact, the authors admit that the reason their recommendations differ from all the others is that other guidelines have not used the GRADE approach. And the reason is “you can’t randomize people to smoke, avoid physical exercise, breathe polluted air, or eat a lot of sugar or red meat and then follow them for 40 years to see if they die. But that doesn’t mean you have no evidence. It just means you look at the evidence in a more sophisticated way.” And alternative approaches to GRADE exist, like, for example, NutriGrade, which have specifically been developed to evaluate evidence from studies of nutritional and lifestyle factors.

So, are the author’s appeals to standards of evidence motivated by a genuine interest in getting to the bottom of it, or just to advance the financial interests of industry, as the same lead author had done previously at the behest of soda and candy companies? “The tool he employed in his meat and sugar studies could be misused to discredit all sorts of well‑established public health warnings, like the link between secondhand smoke and heart disease, air pollution and health problems, physical inactivity and chronic disease, and trans fats and heart disease.” Industries could use it to sow doubt in any field where randomized controlled trials are not feasible, such as climate change. What are you going to have, some placebo planet? Strict adherence to GRADE guidelines could even be used to question the link between smoking and lung cancer.

I know you can’t randomize people to smoke, but can’t you randomize people to quit? Here’s a randomized controlled trial of the effect on aged men of advice to stop smoking. Of those randomized to quit, 13.7 percent died within the study window, whereas of those in the control group who got no special instruction, only 12.9 percent died. In other words, it didn’t work. Disappointingly, the researchers concluded, we find no evidence at all of any reduction in total mortality. Wait, so is smoking not bad for you after all? Of course not. Does anyone see the fatal flaw? They didn’t randomize people to quit. They randomized people to advice to quit smoking. It’s not like they could lock people in a room for a few years. At the last follow-up, the stop smoking group was smoking 8 cigarettes a day compared to 12 cigarettes a day in the control group; so, it’s no surprise there was no difference in mortality since there was hardly any difference in smoking. And the same thing with diet.

There have been massive randomized dietary trials: the Women’s Health initiative, the Multiple Risk Factor Intervention Trial that wasted hundreds of millions of dollars, because people just flat out failed to follow the dietary advice. So, the groups ended up eating similar diets at the end, so had similar disease outcomes, just like the randomized smoking-quit trial. It’s not like the failure was a result of inexperienced investigators; these trials were conducted by some of the very best research teams, who invested enormous efforts to achieve their goals. But it just shows you can’t really run decade-long randomized trials that require changes in eating behavior. People just won’t do it. Randomized controlled trials couldn’t even show an effect on mortality of smoking, which is pretty remarkable, considering that smoking is one of the most powerful known risk factors in the world.

So basically, the foregone conclusion putting any kind of junk to the test in this manner would echo the new meat report conclusion that people should “eat whatever they want and do whatever they want.” It’s like a hijacking of evidence-based medicine. Of course, we want the best evidence possible, but the whole process is now being “manipulated and misused to support subverted or perverted agendas.”

“It’s very exciting and attention-grabbing to say there’s no need to reduce meat intake. It’s less exciting to say we reviewed studies to evaluate the validity using a system not meant to evaluate the validity of these studies, and what we found is nothing.” When asked whether physicians can advise persons whether a salad is healthier than a bowl full of sugar, one of the senior co-authors of the meat papers responded that physicians should tell persons that “the quality of evidence is low; so, it depends almost entirely on their preferences.” “When GRADE criteria do not allow us to strongly recommend against smoking a cigarette with your bowl of sugar, we believe that alternative grading systems are preferable.”

Please consider volunteering to help out on the site.

• Johnston BC, Zeraatkar D, Han MA, et al. Unprocessed red meat and processed meat consumption: dietary guideline recommendations from the nutritional recommendations (Nutrirecs) consortium. Ann Intern Med. 2019;171(10):756-64.
• Karasu SR. A Sacred Cow? Controversial Recommendations About Red Meat. Psychology Today. November 2, 2019.
• LaMotte S. Red and processed meat are OK to eat, controversial new guidelines claim. Don’t believe it, leading experts say. CNN. September 30, 2019.
• Koenig D. Controversial Studies Say It’s OK to Eat Red Meat. WebMD. September 30, 2019.
• Qian F, Riddle MC, Wylie-Rosett J, Hu FB. Red and processed meats and health risks: how strong is the evidence? Diabetes Care. 2020;43(2):265-71.
• Dyer O. New red meat guidelines are undermined by undisclosed ties and faulty methods, say critics. BMJ. 2019;367:l6001.
• New “guidelines” say continue red meat consumption habits, but recommendations contradict evidence. Harvard T.H. Chan. September 30, 2019.
• Dyer O. Food fight: controversy over red meat guidelines rumbles on. BMJ. 2020;368:m397.
• Parker-Pope T, O'Connor A. Scientist Who Discredited Meat Guidelines Didn’t Report Past Food Industry Ties. The New York Times. October 4, 2019.
• Reinhardt S. New Studies on Red and Processed Meat Are a Big, Fat Nothingburger. Union of Concerned Scientists. October 2, 2019.
• Zeraatkar D, Johnston BC, Bartoszko J, et al. Effect of lower versus higher red meat intake on cardiometabolic and cancer outcomes: a systematic review of randomized trials. Ann Intern Med. 2019;171(10):721-31.
• Jukola S. On the evidentiary standards for nutrition advice. Stud Hist Philos Biol Biomed Sci. 2019;73:1-9.
• Erickson J, Sadeghirad B, Lytvyn L, Slavin J, Johnston BC. The scientific basis of guideline recommendations on sugar intake. Ann Intern Med. 2016;166(4):257-67.
• Giovannucci E, Rimm E. Red and processed meat consumption and risk for all-cause mortality and cardiometabolic outcomes. Ann Intern Med. 2020;172(7):510-1.
• Rose G, Hamilton PJ. A randomised controlled trial of the effect on middle-aged men of advice to stop smoking. J Epidemiol Community Health (1978). 1978;32(4):275-81.
• Willett WC. The WHI joins MRFIT: a revealing look beneath the covers. Am J Clin Nutr. 2010;91(4):829-30.
• Satija A, Yu E, Willett WC, Hu FB. Understanding nutritional epidemiology and its role in policy. Adv Nutr. 2015;6(1):5-18.
• Ioannidis JPA. Hijacked evidence-based medicine: stay the course and throw the pirates overboard. J Clin Epidemiol. 2017;84:11-3.

Motion graphics by Avo Media

• animal products
• cancer
• candy
• chronic diseases
• Harvard
• industry influence
• lifespan
• lifestyle medicine
• longevity
• meat
• mortality
• processed meat
• red meat
• smoking
• soda