How I read nutrition-related scientific studies

Does red meat cause cancer? Are eggs good for you? What about coffee? Scientists often seem to flip flop on life’s most important questions. I ran across an article covering the impact of impaired sleep and caffeine on blood glucose control. Given that I consume caffeine regularly and was reading this article at 3 AM, it seemed relevant. The study1 is from the University of Bath (the city in England, not a place of higher education for bathtubs).

Standard internet disclaimer: I’m not a doctor nor do I play one on the internet. Don’t take anything here as medical advice.

Statistical Rigour

What you can conclude from an experiment’s results depends entirely on how the experiment was set up. That is why there are systems like blind studies or control groups to limit the influence of other variables that could impact the outcome. The article looks at a N=29 study done at the University of Bath (the UK city, not a place of higher education for bathtubs). It looked at three scenarios with a subsequent sugar intake and tracked the participants’ blood glucose level.

  1. Undisturbed sleep with no morning coffee
  2. Disturbed sleep with no morning coffee
  3. Disturbed sleep with morning coffee

With this, we can compare the impact of sleep impairment on caffeine-free individuals (1 vs. 2) and the impact of caffeine on sleep-impaired individuals (2 vs. 3). The first thing that stood out was the omission of undisturbed sleep with coffee although we may need to read between the lines here. One nitpick on the methodology is that the study could have tried to control for the act of drinking coffee through decaf coffee or use caffeine pills and a placebo pill. The groups being compared should be as close as possible except for the variable(s) you’re interested in studying.

The study found no statistically significant difference between 1 & 2 and a statistically significant increase in 3 vs. 2. Although the study looked at both blood glucose and insulin levels, I’ll focus on the former for simplicity. Next, I’ll look at statements or conclusions made by the authors of the article and the study.

New Learnings and Hidden Assumptions

'Though, the study was limited and further research is needed into the effects of caffeine first-thing in the morning on the metabolism, the early findings suggest drinking coffee first could limit the body’s ability to process sugar immediately after.'

Some quotes from the authors of the study seem to indicate that they wanted to explore the impact of sleep impairment on metabolism but were unable to find any difference (1 vs. 2). While I think that the absence of statistically significant differences can be very important, it is certainly less exciting to talk about.

Instead, the narrative focused more on the consumption of coffee before breakfast. But has that been explored already? A minute of Googling showed a study published in 1998 showing increased blood glucose levels corresponding to blood caffeine levels so this particular conclusion seems less interesting.


'Put simply, our blood sugar control is impaired when the first thing our bodies come into contact with is coffee especially after a night of disrupted sleep. We might improve this by eating first and then drinking coffee later if we feel we still feel need it.' -Professor James Betts, who oversaw the study

Based on the study methodology, the conclusions that we can draw would be on the impact of coffee only after a night of disrupted sleep, not especially (2 vs. 3). To use that word, the study would have had to look at a group with undisturbed sleep and coffee. Then you can measure the impact of sleep impairment and coffee separately as well as see if there are any synergistic effects.

[Warning: opinion] While eating before coffee sounds like sensible advice, there is a hidden assumption here that we want to flatten the blood sugar curve as much as possible. Hyperglycemia leads to diabetes so high glucose levels is bad for you, right? If that’s the case, why aren’t all athletes who eat voraciously suffering from diabetes? If your goal is to avoid blood sugar spikes, I would suggest consuming no calories (please don’t do this - you will die). I believe the jury’s still out on long-term health effects of acute elevation of blood glucose.

Conclusion

When confronted with complex topics, most people replace the difficult question of “does impaired sleep combined with caffeine compromise blood glucose control” with an easier one such as “is coffee bad for you”2. Besides being misinformed, perhaps a more significant consequence of this substitution heuristic is that the general public believe that scientists can’t agree on anything and end up losing faith in the scientific community.

So next time you read through a nutrition-related article, I would recommend that you add a healthy dose of skepticism, look at the actual paper, read what is & is not being said, and draw your own conclusions. It is more work than parroting sound bites but you will get a lot more out of it.

1Smith, H., Hengist, A., Thomas, J., Walhin, J., Heath, P., Perkin, O., . . . Betts, J. (2020). Glucose control upon waking is unaffected by hourly sleep fragmentation during the night, but is impaired by morning caffeinated coffee. British Journal of Nutrition, 124(10), 1114-1120. doi:10.1017/S0007114520001865
2This is known as substitution bias which is discussed in Daniel Kahneman’s book Thinking, Fast and Slow