If People Who Take It Live Longer, How Were They Different to Begin With? — Confounding

The Headline "People Who Take Them Live Longer"

"People who take nutritional supplements every day live longer." You frequently encounter sentences of this shape in health news. It is a story where researchers followed people who took them and people who did not for a long time, and found that fewer people passed away on the side that took them.

Let us translate this into numbers. Researchers followed 100 people who took nutritional supplements every day and 100 people who did not for 10 years. Among the 100 people who took them, 10 passed away, while among the 100 people who did not, 20 passed away. The risk of death for the group that took them is half that of the group that did not, and the difference is 10 people per 100. This arithmetic was covered in "If the Risk Was Cut in Half, How Many Fewer People Got Sick? — Relative and Absolute Risk".

When people hear these numbers, they usually take one step further. They think the supplements saved 10 people. However, the sentence that people who take them died less and the sentence that the supplements saved people are different sentences. The former sentence is the result of counting, and the latter sentence is a claim about the cause. This article deals with what can sneak in between those two sentences, and what you can ask when you encounter such a headline.

This supplement and these 200 people are examples made to aid understanding, and are not the results of an actual product or study. We will carry this example with us to the end of the article. The names of drugs, vaccines, and hormones appearing later are merely examples showing how the numbers of actual studies are read, not judgments on whether to use them or not.

They Were Different People to Begin With

Let us divide the same 200 people again this time based on exercise. Among the 100 people taking the supplement, 80 people exercise regularly, and 20 people do not exercise. Among the 100 people not taking it, 40 people exercise, and 60 people do not exercise.

Looking only among the people who exercise, at the rate of 1 out of 20 passed away regardless of whether they took the supplement or not. 4 out of 80 and 2 out of 40 are the same ratio. Looking only among the people who do not exercise, both are at the rate of 3 out of 10. 6 out of 20 and 18 out of 60 are also the same ratio.

When comparing people with the same exercise habits, the difference between the side that took the supplement and the side that did not disappeared. Even so, the totals are 10 people versus 20 people. This is because two things overlapped. People who exercise passed away much less than people who do not exercise. And 80 people who exercise gathered in the group taking the supplement, while 40 gathered in the group not taking it.

This example was constructed so that the supplement does nothing. Exercise is what determined death, and people who exercise just happened to take better care to take the supplement as well.

An association is a relationship where one appears more or less when the other is present. In this example, there is clearly an association between taking the supplement and living a long life. 10 people and 20 people are not numbers that were counted incorrectly. However, having an association and the supplement being the cause are different stories.

A confounding factor is a third factor that influences both the side that looks like a cause and the outcome. In this example, the side that looks like a cause is the supplement, the outcome is whether they are alive 10 years later, and the third factor is exercise. Exercise reaches both whether they will take the supplement and whether they will be alive 10 years later. When there is a factor that moves together with both sides like this, the relationship seen between the two may be due to that factor instead of the side that looks like the cause (PMID 38524951, PMID 28817531). This appearance of the relationship being skewed like this because of a third factor is called confounding.

Just because they move together does not make it the cause. The group that takes it and the group that does not may have been different people from the start.

"People Who Were Already Different" in Real Research

In the 200-person example, each person decided for themselves whether to take the supplement. The researchers did not split the groups; they just watched and counted.

An observational study is research where the researchers do not split the groups, but simply let people live as they already do, watching and counting. Because the division into groups is decided by the people's own choices or a doctor's prescriptions, the people who ended up in one group versus the other may have been different from the very beginning.

In real observational studies, the ways in which two groups differ from the start appear in largely two directions. The first is when people who take care of themselves gather in one group. The healthy user effect is a phenomenon where people who steadily look after something for prevention are also ahead in other behaviors or their baseline circumstances, making that preventive measure look more beneficial than it actually is. The exercise in the 200-person example falls right into this spot. In observational studies examining the effects of preventive measures, biases like this—meaning results skewed in one direction due to people's behaviors or baseline traits—have been identified across multiple lines (PMID 21203857).

The second is the opposite direction. People who need medication take it. In an observational study, who receives treatment is not decided by drawing straws, and the reason they ended up receiving treatment can be intertwined with the risk of future disease. Then, the group that received treatment and the comparison group differ in risk from the start, and the results can be skewed (PMID 11941379).

In one survey, people who newly started a medication were sicker even before starting it (PMID 12022317). This was a survey where arthritis patients treated by 433 American rheumatologists were given questionnaires across two occasions. In between, 1,517 people who newly started a newly released anti-inflammatory painkiller were compared with 5,120 who did not start it. The people who newly started had a greater history of experiencing drug side effects, had more severe pain and difficulty using their bodies, and utilized hospitals more. The authors summarize that the overall severity measured by multiple scales was about 25% higher.

Therefore, the direction in which confounding twists the results is not just one-sided. When people who take care of themselves gather, a preventive measure can look more beneficial than it actually is, and when sicker people take a drug, the drug can look more harmful than it actually is.

If what split the two groups in an observational study was the people's own choices, the reason that drove those choices may also be tied to the outcome.

If You Divide by Drawing Lots

Let us return to the 200 people. This time, instead of letting everyone choose, we draw lots to assign 100 people to take a nutritional supplement and 100 people not to.

Among the 200 people, 120 exercise. Drawing lots generally divides the people who exercise roughly in half. Let us suppose that each side ends up with 60 people who exercise and 40 who do not. If we follow the exact percentages from the previous table, on both sides 3 out of 60 and 12 out of 40—for a total of 15 out of 100—pass away. The difference is 0. The correct answer for this example, where the nutritional supplement did nothing, comes out just like that. In reality, drawing lots does not divide people in half so neatly, but merely makes them roughly similar, so the actual numbers wobble a little around 0.

A test divided by drawing lots in this way is called a randomized trial. Neither the people nor the researchers choose who goes into which group; it is left to chance.

The power of drawing lots does not end with exercise. Whether it is dietary habits or sleep, differences that the researchers did not even think of go through the same drawing of lots. Therefore, in groups divided at random, differences that could become confounding factors—whether observed or unobserved—are distributed sufficiently evenly between both sides. This is why confounding is generally not a problem in randomized trials. Conversely, in observational studies, confounding can be a major problem (PMID 28817531). When properly applied, simply dividing people at random is expected to make most associations that arose without a causal relationship disappear (PMID 20335814).

Drawing lots distributes even differences that researchers did not think of roughly evenly to both sides. That is why randomized trials answer the question "Is it the cause?" more directly.

There is a reason for the word "roughly." Even when divided by drawing lots, a few more people who exercise could end up on one side. How large such a wobble of chance can be is the domain of "What Does "Statistically Significant" Actually Say? — The p-Value, Explained with a Coin Toss" and "How Much Does a Number Counted Once Wobble? — The Two Numbers in the Parentheses, the Confidence Interval".

There are also cases where observational studies and randomized trials gave different answers. Among observational studies covering preventive measures, several widely known studies reported inflated relationships, and those results were later contradicted by randomized trials (PMID 21203857).

A widely known example involves hormone therapy in women before and after menopause and myocardial infarction. Looking at a review paper summarizing 50 papers on this topic, the answers from the two sides pointed in different directions. Many observational studies pointed to hormone therapy preventing myocardial infarction. However, in many of those studies, the results were not statistically distinct. This means they did not exceed the range that could be explained by the wobble of chance. On the other hand, one large randomized trial that examined the disease-preventing effect in generally healthy women reported a small but statistically distinct increase in risk from therapy using two hormones combined (PMID 15924502).

The authors of the same review consider the evidence on whether the risk changes depending on the duration of use to be unclear as well. Therefore, they write that the idea that the protective association pointed to by observational studies is the cause is not supported. That does not mean they accepted the results of the randomized trial as is either. The authors cite two other explanations. One is the bias that arises on the side where disease is more easily detected, and the other is confounding that remains even after filtering it out through calculation. Both explanations apply to the association seen in the randomized trial as well as the association seen in the observational study. The conclusion is that there is insufficient evidence to say that hormone therapy is associated with a change in the risk of myocardial infarction in most women. They add the clue that a group of women with specific genetic differences may be more sensitive (PMID 15924502).

Calculation-Based Matching — You Can Only Match What You Measured

That does not mean every question can be solved by a draw of lots. During pregnancy, randomized trials are very difficult. Therefore, when evaluating the safety of medications during pregnancy, observational studies are frequently the only option (PMID 36805380). Randomized trials are expensive, can be difficult to execute, and have limitations in generalizing results beyond the people enrolled in the trial (PMID 42175748).

Observational studies have a role that randomized trials cannot fulfill. For certain questions, they may be practically the only source of information, they can deal directly with the people and situations receiving actual medical care, and they also help find treatments worth investigating through randomized trials. A well-conducted randomized trial remains preferred for asking whether something is a cause. However, it has been suggested that under strong assumptions, the results of a well-conducted observational study can also be interpreted as cause-and-effect relationships (PMID 38722735). Observational studies remain useful, and at times are the only evidence.

Observational studies therefore handle confounding through calculation. Comparing people who exercise separately from those who do not exercise within a sample of 200 people is the simplest form of that method.

Adjustment is the work of matching through calculation as if comparing people with identical confounding factors. Factors suspected of being confounders must be identified, measured, and matched in the calculation (PMID 38524951). Several calculation methods are used to measure the characteristics of patient groups and match the measured confounding factors (PMID 37783401).

There are examples where adjustment did its job. This was a study that interviewed 3,304 local residents selected through random digit dialing multiple times at 2-week intervals (PMID 18840922). Among people who initially had no depression, those taking antidepressants developed new depression at a rate 3.9 times higher than those not taking them. This is a figure that makes the medication appear to cause depression.

The authors adjusted for symptom scores during the 2 weeks immediately preceding the onset of new depression. Consequently, this multiple shrank to 1.2 and became statistically non-significant. The authors conclude that antidepressant use is entangled with symptom severity, and therefore observational studies where antidepressants appear harmful can be biased (PMID 18840922). The authors viewed symptom severity as a confounding factor—specifically, a factor connected to both whether someone takes the medication and whether depression develops. Once that was measured and matched, the figure that was 3.9 times higher came close to 1.

Adjustment comes with one condition. You can only match what you measured. Suppose there was one more hidden difference of dietary habits in the 200-person example. If people taking nutritional supplements also pay closer attention to their meals, but the researcher did not ask about dietary habits, dietary habits do not enter the calculation. If exercise was only asked in two categories ("yes" or "no"), people who run every day and those who occasionally walk are mixed within the same "yes." If a difference remains even after adjusting for exercise, this data cannot separate whether it is thanks to the supplements or due to such remaining differences.

Residual confounding is confounding that remains even after making adjustments. While observational studies can incorporate confounding factors to make adjustments, they cannot eliminate residual confounding stemming from inaccurately measured factors, unmeasured factors, or factors nobody knows (PMID 31365194). Therefore, the possibility that unmeasured confounding factors remain cannot be completely ruled out (PMID 33826756).

Remaining confounding may not be small. There is a study that estimated its magnitude using computer simulation (PMID 17615092). By creating situations on a computer where something is actually not a cause, the study examined what numbers emerge if confounding factors are crudely measured or not measured at all. Under plausible assumptions, effect sizes of the magnitude commonly reported in observational studies investigating the causes of disease (observational epidemiology studies) could be produced solely by such remaining confounding. This result came from calculations built on assumptions, not from data observing actual humans.

Tracks of this are visible in actual data as well. This is a systematic review that gathered and examined 23 observational studies on influenza vaccine effectiveness using a prescribed, comprehensive method (PMID 26474974). The starting point is the fact that during seasons when influenza is barely circulating, there are also few flues for the vaccine to prevent. If vaccinated people show better outcomes even in those seasons, that difference is difficult to attribute to the vaccine. The authors viewed this as a signal of residual confounding. Even after making adjustments, such a signal was clear in 9 studies.

In the same review, adjustment actually inflated the vaccine effectiveness estimates (the degree varied with each measured outcome). The authors believe that the two directions of bias seen earlier operate together in these studies. Confounding arising from being sick enough to get vaccinated is corrected to some extent by adjustment. However, bias stemming from healthy people being proactive about vaccination remains—at least as long as one measures broad outcomes like all-cause mortality, which counts all deaths including those unrelated to influenza. Therefore, they recommend no longer measuring influenza vaccine effectiveness using studies that employ such broad outcomes, namely cohort studies following people through records accumulated for administrative purposes (PMID 26474974). This conclusion pertains to one method of measuring effectiveness. This review does not judge whether influenza vaccine is effective.

Researchers also know this problem and perform calculations. There is a method to calculate how strongly an unmeasured confounding factor must be entangled with both the treatment and the outcome in order to completely explain away an observed association. If that value is large, it means a hidden factor must be quite strong to completely erase that association. The proponents suggest that all observational studies seeking evidence of causation should report this value or other sensitivity analyses—calculations showing how robust the result is to unmeasured confounding (PMID 28693043).

P-values and confidence intervals do not filter out confounding either. This is because these two numbers deal with random fluctuation. Even if fluctuation is small because the sample is large, estimates can be precisely wrong if confounding is present (PMID 38972732). If a statistically distinct association emerges even though the true relationship is not causal, bias and confounding are common suspects (PMID 28817531). Therefore, even if the p-value is small and the confidence interval is narrow, one must ask separately: "Would they have been different to begin with?"

Adjustment works only on what was measured. Unmeasured differences may remain.

When You Read a News Headline

"People Taking a New Supplement Cut Their 10-Year Risk of Death in Half." Let us imagine you encounter a headline like this. It is the same kind of number as the 200 people in this article. From what we have seen so far, there are three questions you can ask.

The first question is, 「How were those people different to begin with?」. If someone takes it upon themselves to look after and take a supplement, they might also be someone who looks after their exercise and diet, and the healthy user effect is right there. Conversely, if many people started taking it because their bodies were unwell, the supplement might look worse than it actually is.

The second question is, 「Who divided them?」. If people chose for themselves whether to take it, it is an observational study, and the first question remains fully intact. If they were divided by a draw of lots, it is a randomized trial, and even differences you did not think of are distributed roughly evenly across both sides. If there is a well-conducted randomized trial addressing the same question, that side gives a much more direct answer to 「Is it the cause?」 (PMID 38722735).

The third question is, 「What was adjusted for, and what was not?」. If an article only notes that 「age and sex were adjusted for」, the exercise from the 200-people example has not been adjusted for yet. Even if it states that exercise was adjusted for, residual confounding like dietary habits or the amount of exercise can still remain.

Papers do not always write down these questions first. There is a survey that randomly selected and examined 120 observational studies published in high-impact journals between 2011 and 2012 (PMID 28943377). Of those, 68 papers mentioned confounding, but only 2 papers explicitly stated that the results should be interpreted cautiously because of confounding. The authors conclude that even when mentioning confounding, researchers are usually convinced that it has little to do with their own results. They also report that papers more cautiously acknowledging the effect of confounding did not suffer in citations. This means that the mere appearance of the word confounding in a paper does not imply that the authors factored it into the interpretation of their results.

The two questions raised in If the Risk Was Cut in Half, How Many Fewer People Got Sick? — 「Out of how many?」 and 「For how long?」 — are still put to use. If you found the answers 「from 20 people down to 10 out of 100」 and 「over 10 years」 in the article, the magnitude is 10 people per 100 over 10 years. Those two questions tell you the magnitude of the difference, and the question in this article asks whether those 10 people truly came from the supplement.

The answer 「They might have been different to begin with」 does not mean 「The supplement has no effect」 right away. It means that whether it is the cause remains an open question based on this article alone.

When you encounter the headline 「People Who Take It Live Longer」, the very first question to ask is 「How were those people different to begin with?」.

Where to Look Further

How to read the numbers listed side by side in an adjustment table, and what criteria are used to select the variables to adjust for, are covered in What "Adjusted for Age, Sex and BMI" Actually Did — How to Read Confounding and Adjustment. How to gauge the confounding that remains even after adjustment is then looked at in What Is Left After Adjustment.

Adjusting for anything does not always make things better. Applying the method for correcting confounding to a variable in the wrong place can itself create bias (PMID 34939918). That place is covered by The Variables You Must Not Adjust For.

Other methods that try to determine causation without a draw are covered in Causal Inference — How to Establish Causation Without Experiments, and the limitations of randomized trials are covered in Is the Randomized Controlled Trial Really the 'Gold Standard'?.

Other articles in Numbers from Scratch are If the Risk Was Cut in Half, How Fewer People Got Sick? asking how many people correspond to a "50% risk reduction", What Does "Statistically Significant" Actually Say? asking whether a result is by chance, and How Much Does a Number Counted Once Wobble? noting how much a number can wobble.

Source Papers

• "Confounding by Indication, Confounding Variables, Covariates, and Independent Variables: Knowing What These Terms Mean and When to Use Which Term." (2024) — PMID 38524951 — A confounding variable is associated with both the independent variable and the outcome, meaning the discovered relationship may exist because of the confounder rather than the independent variable. Potential confounders must be identified and adjusted for in multivariable regression.
• "Bias, Confounding, and Interaction: Lions and Tigers, and Bears, Oh My!" (2017) — PMID 28817531 — A confounding variable is a factor associated with both the exposure and the outcome. Randomly assigned groups are sufficiently balanced in both observed and unobserved confounders so confounding is generally not an issue, but it can be a major problem in observational studies. Bias and confounding are common potential explanations for a significant association when the true relationship is not causal (review).
• "Healthy user and related biases in observational studies of preventive interventions: a primer for physicians." (2011) — PMID 21203857 — Several widely known observational studies of prevention reported inflated relationships only to be refuted by randomized controlled trials. It outlines biases entangled with patient behaviors and characteristics (such as the healthy user effect) and discusses design and analysis methods to reduce them (editorial).
• "Confounding by indication in epidemiologic studies of commonly used analgesics." (2002) — PMID 11941379 — In observational studies, treatment assignment is not random, and the reason for treatment can be related to the risk of future health outcomes, leading to biased results due to baseline risk imbalances between the treatment and comparison groups.
• "Increase in lifetime adverse drug reactions, service utilization, and disease severity among patients who will start COX-2 specific inhibitors…" (2002) — PMID 12022317 — Among arthritis patients treated by 433 US rheumatologists, the 1,517 patients newly starting COX-2 inhibitors had more history of adverse effects, pain, disability, and service utilization than the 5,120 who did not start them, with an overall severity roughly 25% higher. It recommends reflecting these variables using propensity scores.
• "Negative controls: a tool for detecting confounding and bias in observational studies." (2010) — PMID 20335814 — Random assignment, when properly applied, is expected to eliminate most associations that are not causal. It suggests making greater use of negative controls in observational studies.
• "Hormone replacement therapy and risk of acute myocardial infarction: a review of the literature." (2005) — PMID 15924502 — Review of 50 papers. Numerous observational studies showed a protective direction (many not significant), while a large primary prevention randomized trial in healthy women (WHI) showed a small but significant increase in combination therapy. Because evidence on duration of use is uncertain, it does not support the causality of the protective association, concluding that detection bias and residual confounding are alternative explanations for both associations, and evidence is insufficient to speak of an association with risk change in most women (with possible exceptions for specific genetic polymorphism subgroups).
• "Emulating Target Trials to Avoid Immortal Time Bias - An Application to Antibiotic Initiation and Preterm Delivery." (2023) — PMID 36805380 — Randomized trials during pregnancy are extremely difficult, making observational studies often the only choice.
• "A Practical Guide to Target Trial Emulation: Connecting Randomized Trials and Real-World Data in Cardiovascular Research." (2026) — PMID 42175748 — Randomized controlled trials are the standard for causal inference, but they have limitations in cost, feasibility, and generalizability. Target trial emulation increases validity but does not eliminate residual confounding, and it complements rather than replaces randomized evidence (editorial).
• "Causal Inference About the Effects of Interventions From Observational Studies in Medical Journals." (2024) — PMID 38722735 — Although well-conducted randomized trials are the preferred method for causal questions, causal interpretation of well-conducted observational studies may be possible if strong assumptions hold. Observational studies may be the only practical source of information, deal with real clinical populations, and help identify subjects for subsequent experiments. It proposes a six-question framework.
• "The use of the E-value for sensitivity analysis." (2023) — PMID 37783401 — Observational studies lack random assignment, increasing the potential for confounding, which is reduced as propensity score matching and regression adjust for measured confounding factors. Unmeasured confounding is assessed through sensitivity analysis.
• "Confounding by severity and indication in observational studies of antidepressant effectiveness." (2008) — PMID 18840922 — 3,304 community residents: antidepressant use among initially non-depressed individuals was associated with a depression incidence HR of 3.9, but dropped to 1.2 (not significant) when adjusted for symptom scores from the prior two weeks. It concludes that antidepressant use is entangled with symptom severity, and observational studies that look harmful are biased because of that.
• "Antidepressant Drugs and the Risk of Hip Fracture in the Elderly: Is There More to It Than Confounding by Indication?" (2019) — PMID 31365194 — Observational studies can adjust for confounding variables, but cannot eliminate residual confounding coming from inaccurately measured, unmeasured, and unknown factors (commentary).
• "When Can Nonrandomized Studies Support Valid Inference Regarding Effectiveness or Safety of New Medical Treatments?" (2022) — PMID 33826756 — Non-randomized studies are far more susceptible to the effects of confounding, and the existence of unmeasured confounding can never be completely ruled out. It presents a question framework to assess reliability.
• "The impact of residual and unmeasured confounding in epidemiologic studies: a simulation study." (2007) — PMID 17615092 — Simulation: under plausible assumptions, effect sizes commonly reported in observational epidemiological studies can be generated solely by residual and unmeasured confounding.
• "Frequency and impact of confounding by indication and healthy vaccinee bias in observational studies assessing influenza vaccine effectiveness: a systematic review." (2015) — PMID 26474974 — 19 of 23 papers had a high risk of bias. Adjustment increased effect estimates (12% for all-cause mortality, 9% for hospitalization, 7% for influenza-like illness), and 9 papers still showed significant effects outside the influenza season even after adjustment, suggesting residual confounding. It concludes that while adjustment somewhat fixes confounding by indication, healthy vaccinee bias remains in non-specific outcomes like all-cause mortality, so administrative data cohorts using such outcomes should not be used to measure influenza vaccine effectiveness.
• "Sensitivity Analysis in Observational Research: Introducing the E-Value." (2017) — PMID 28693043 — The E-value is the minimum strength of association that an unmeasured confounding factor must have with both the treatment and the outcome to fully explain the observed association. It suggests that all observational studies aiming to establish causal evidence report this or another sensitivity analysis.
• "Methodological Tutorial Series for Epidemiological Studies: Confounder Selection and Sensitivity Analyses to Unmeasured Confounding…" (2025) — PMID 38972732 — Even if a sample is large and variance is small, systematic errors like confounding can lead to precisely inaccurate estimates (tutorial).
• "Interpretation of epidemiologic studies very often lacked adequate consideration of confounding." (2018) — PMID 28943377 — Random sample of 120 cohort and case-control studies in high-impact journals from 2011–2012: 68 papers mentioned confounding, but only 2 used caution/cautious because of it. The authors conclude that even when mentioning confounding, authors tend to be confident that it is irrelevant to their results, and cautious acknowledgment was not associated with citation penalties.
• "Collider Bias in Observational Studies." (2022) — PMID 34939918 — Applying methods that correct for confounding to a collider variable can itself create bias, so confounding variables and collider variables must be distinguished.