How Much Does a Number Counted Once Wobble? — The Two Numbers in the Parentheses, the Confidence Interval

The Parentheses at the End of the Title

"New preventive medicine cuts heart disease risk in half." This is a sentence you have likely seen in a newspaper headline. When you ask two questions of such a headline—"Out of how many?" and "For how long?"—the half reveals itself as a concrete number of people. These are the two questions covered in "If the Risk Was Cut in Half, How Many Fewer People Got Sick? — Relative and Absolute Risk".

Let us suppose the answers to those two questions are written in the article. Among people who did not take the preventive medicine, 2 out of 100 fell ill over the course of one year; among those who took it, 1 out of 100 fell ill. Scaled up to 1,000 people, that is 20 and 10, and the difference is 10 fewer per 1,000 people.

In a paper's abstract, one more set of parentheses is attached behind this number. If it was a study that recruited 10,000 people on each side, it looks something like this: "10 fewer fell ill per 1,000 people (95% confidence interval 7–13 people)". These two numbers inside the parentheses, 7 and 13, are the protagonists of this article.

The two numbers inside the parentheses show how much the number obtained from a single study can wobble. This article covers in turn where that wobble comes from, what it means for the parentheses to be wide or narrow, and what you should ask when you encounter parentheses.

The preventive medicine and numbers in this article are examples created to aid understanding and do not represent the results of any actual study. We will carry this example with us all the way to the end of the article. The real-world studies that appear later serve as examples of how parentheses appear in actual papers.

If We Had Gathered Different People, We Would Have Gotten Different Numbers

First, let's look at a small example where 100 people were gathered. Among 100 people who did not take the preventive medicine, 2 people fell ill over the course of a year. These 2 people are the count measured directly from these 100 people.

What if a different 100 people had been gathered and watched for the same year? Even if it is truly a disease that strikes 2 out of 100 people, the newly gathered 100 people could yield 0 cases or 4 cases. If this article runs the math, the probability of getting 0 cases is over one in ten, and the probability of getting 4 cases is also close to one in ten. The numbers fluctuate depending on who happens to be selected.

To talk about this fluctuation, two names are needed. The sample is the people the study actually gathered and measured. The true value is the number that would emerge if the study could count the entire target population it wants to know about. The 「2 out of 100 people」 from earlier is the number that came from the sample. Because we cannot count the entire population, nobody gets to see the true value directly.

So what we get by studying a sample is only an estimate of the true value (PMID 19114737). Even if a study is repeated in an identical manner, the magnitude of the effect can vary considerably from sample to sample. That is true for sample sizes common in psychology research (PMID 32378428). A difference that looked hazy and leaned to one side in a small study—in other words, a 「trend」—can disappear or reverse direction when the sample is made larger (PMID 15526517).

The number a single study puts out is the number from a single sample, and had it been a different sample, it would have yielded a slightly different number. Therefore, receiving just a single number makes it impossible to know how far that number might be from the true value.

The Parentheses Are the Width of the Swing

Let us return to the parentheses at the beginning. This time, suppose it is a study that gathered 10,000 people who took a preventive drug and 10,000 people who did not, and followed them for 1 year. Among those who did not take it, 200 people got sick, and among those who took it, 100 people got sick. Translated to per 1,000 people, that is 20 people and 10 people, and the difference is 10 people per 1,000 people.

The "95% confidence interval 7–13 people" attached behind this 10 people is the parentheses. A confidence interval is a range of values that do not contradict the data, written alongside the single value produced by a study. "Do not contradict" means that even if the true value was any value within that range, this result could easily happen purely due to the swing of the sample.

In this example, whether it is a drug that makes 7 fewer people per 1,000 get sick, or a drug that makes 13 fewer people get sick, neither contradicts these data. The parentheses list the values that fit the data in this way, and their width tells us how certainly we know the result (PMID 38219106).

The 10 people in the middle is the figure actually obtained from this sample. The 7 people and 13 people at either end are the positions widened by taking into account how much the sample could have swung. Therefore, the parentheses tell us two things together. The position where the parentheses are placed tells us which way and by roughly how much the effect is, and the width of the parentheses tells us how certainly we know it (PMID 9172963).

There is a name attached to this certainty. Precision refers to how much less a number swings; if the interval is narrow, it is precise, and if it is wide, it is imprecise. The narrower the parentheses, the more precise the estimate (PMID 9172963).

Even with the same middle number, the parentheses can be different. Let us place side by side three studies that had the same proportion of people getting sick and differed only in the number of people gathered. All three are examples created by this article. The parentheses are values estimated by this article using a commonly used calculation method, and depending on the calculation method, the end digits may vary by about 1 person. For the row where 100 people each were gathered, the numbers varied by dozens of people depending on the calculation method, so no numbers were written.

The middle number is 10 people per 1,000 people in all three rows. The only thing that differs among the three rows is the number of people gathered, and because of that, the width of the parentheses changed. The parentheses of the study that gathered 100 people each are wide enough to span from the drug causing dozens more people to get sick to preventing dozens of people.

Even if the middle number is the same, if the parentheses are wide, it is less certain, and if they are narrow, it is more certain.

The More People, the Narrower the Parentheses

The width of the parentheses is the distance between both ends. In the table above, when the number of people gathered on one side was increased 10-fold from 1,000 to 10,000, the width of the parentheses shrank from 22 people to 6. The number of people became 10 times larger, but the width shrank to below a third. The width does not shrink as much as you increase the people. To get narrow parentheses, you have to gather that many more people.

There are several factors that determine the width of the parentheses (PMID 25742216). Among them, the number of people to gather is what researchers can determine most directly. There are even methods out there to calculate in advance how many people to gather so that the parentheses become as narrow as desired (PMID 12971200).

However, the criterion researchers commonly use to determine the number of people is different. It is the probability of catching an effect when it is there. The number of people determined according to this probability and the size of the expected effect is often too small. This means it falls short of giving parentheses narrow enough to precisely estimate the true value (PMID 21336224).

In actual surveys, too, the parentheses for items with many people to count tended to be narrow. However, the difference was not solely due to the number of people. This is the result of comparing the parentheses of items (indicators) measuring what young children eat across 203 demographic surveys in refugee settlements (PMID 40803324).

The indicator measuring whether a child has ever been breastfed counts all children under two years old who were surveyed. The parentheses for this indicator spread out by about 3.1 percentage points up and down from the middle number (the median value when the 203 cases are lined up). The authors note as the reason that the sample was relatively large and the proportion was close to 100%.

The indicator measuring whether infants under 6 months old were exclusively breastfed counts only some of the children under two years old. The parentheses for this indicator spread out by about 12.6 percentage points up and down, falling among the widest. The authors conclude that because precision is low, one must be extremely careful when reading past survey results for indicators like this.

There are fields where studies with few people are common. Studies on the accuracy of screening tests that sort out people likely to have depression are like that. In a review examining these studies published between 2018 and 2021, there were 103 papers that reported parentheses or from which they could be calculated. Among them, in 58 papers, the width of the parentheses attached to the rate at which the test catches depression was 21 percentage points or more (PMID 35362161). This means the difference between when you take the catch rate at its smallest and when you take it at its largest spans that wide.

The authors conclude that most studies had too few people to produce a precise estimate, and note that the results were similar to previous surveys. The width of the parentheses is mostly determined by the number of people gathered, and if there are few people, the parentheses are wide.

Does the Parentheses Span Beyond "No Difference"?

Let's look again at the parentheses from our example of 1,000 people each. It ranges from 1 more person getting sick per 1,000 people to 21 fewer people getting sick. This set of parentheses contains "no difference"—in other words, 0 people. This means that everything from the drug being slightly harmful to quite beneficial is consistent with this data.

Whether "no difference" falls inside the parentheses tells the same story as the p-value. The p-value is the probability that, assuming there is no difference at all, you would get a result as extreme as or more extreme than the current one. This is the number we saw in "What Does "Statistically Significant" Actually Say? — The p-Value, Explained with a Coin Toss", and when this value is small, we write that the result was "statistically significant".

In common calculations, the two line up like this. If the 95% parentheses include "no difference", the p-value is greater than 0.05. If it bypasses "no difference", the p-value is less than 0.05. That is why you can tell whether a finding is statistically significant just by looking at the parentheses (PMID 25742216). Our earlier parentheses contain 0, so it is on the non-significant side.

The parentheses go one step further here. The method of deciding whether there is a difference using a p-value—that is, significance testing—only asks whether the assumption of no effect is tenable. It does not estimate the size of the effect. Recommendations to report the size of the effect alongside the parentheses instead of significance testing have existed for a long time (PMID 9012613). The reason is that reporting it that way captures all the information the testing gave, while laying bare the size of the effect.

When we move this to a study of 1,000 people each, it looks like this. At its smallest estimate, the drug causes 1 more person per 1,000 to get sick, and at its largest, it causes 21 fewer people to get sick. If you read this result as "there was no effect," the 21-person end disappears. "This data has not yet narrowed it down" is a closer reading.

There are also proposals that turn this reading into a rule (PMID 19114737). The suggestion is to read the result as "inconclusive" if the parentheses span both a meaningfully beneficial value and a meaningfully harmful value, and otherwise to read it toward the observed side. In this proposal, you must first decide what counts as a meaningful magnitude.

The parentheses of 7 to 13 people in the study of 10,000 people each do not contain 0. Even at the smallest estimate, 7 fewer people get sick per 1,000 people. Because the range is narrow and both ends are on one side, both the direction and the magnitude are quite clear.

If the parentheses bypassed "no difference," the next question is whether the smallest estimate is large enough to matter to people. The values inside the parentheses tell you whether there is room for the result to actually matter (PMID 38219106).

Parentheses written as division have one different reading position. In numbers where one proportion is divided by another (relative risk), such as "risk is 0.5 times," if the two sides are the same, the divided value becomes 1 time. Therefore, in these parentheses, "no difference" is 1, and you look to see whether it includes 1.

The p-value and parentheses are usually used together. Interpreted properly, the p-value also gives useful information. However, the conclusion of one review is not to use the p-value as the sole criterion for judgment, but to transparently report the effect size, parentheses, and context (PMID 41250654). The parentheses side also has limitations. A 1997 overview article noted that the risk of misreading even the method of reporting with parentheses does not disappear. It added that because there is still no consensus on which method is better, some experts suggest reporting both (PMID 9172963).

Whether the parentheses contain "no difference" usually tells the same story as the p-value, and the two ends of the parentheses tell you more about what the effect would be at its smallest and largest estimates.

What Is the 95% Attached To?

The 95% in front of the parentheses also has a name. Before the name, let us see how this 95% is made.

Imagine that 100 research teams each repeat a study with 10,000 people using a brand-new sample. Because samples fluctuate, the middle number and the parentheses come out slightly different for each team. Some parentheses catch the true value between their two ends, while other parentheses miss the true value. The method of calculation that makes about 95 out of those 100 parentheses contain the true value on average is the 95% confidence interval.

The confidence level is the 95% attached in front of the parentheses, meaning that if you repeat a study endlessly in the same way, 95% of the parentheses made that way will contain the true value. The "95% confidence level" attached to the end of a public opinion poll article is also this 95%.

A common misunderstanding is attaching this 95% to a single set of parentheses right in front of your eyes. It is like reading it as, "There is a 95% probability that the true value is between 7 people and 13 people." In the thinking behind this calculation method, the true value is a single number that does not move, and the part that moves with each sample is the parentheses. A single set of parentheses right in front of your eyes has already been drawn, and it either contains the true value or it does not. Which one it is, we cannot know.

The 95% is a number attached to how often the method of making parentheses contains the true value, and it does not tell you whether a single set of parentheses right in front of your eyes has contained the true value. This is about all you can say with a single set of parentheses. These parentheses were made by a method that usually contains the true value, and the values inside them do not contradict this data.

This distinction has been reported to be not easy even for experts. One survey gave 120 psychology researchers and 442 students six sentences interpreting confidence intervals and asked whether they were true or false (PMID 24420726). According to the researchers' classification, all six sentences were incorrect interpretations. Yet both researchers and students chose more than three sentences on average as correct. The researchers were hardly better than the students who had never learned statistical inference.

Objections were raised against this result. The first argument of one critical paper was the classification of the sentences. There are two interpretations of confidence intervals, so you cannot conclude that someone misunderstood just because they agreed with a certain sentence. The second argument was the design. It is possible this result was produced because people were made to answer all the sentences.

The authors of this critical paper re-surveyed by adding two sentences containing correct interpretations. First-year students who had not yet learned chose correct and incorrect sentences at the same rate. The authors take this to mean that the two kinds of sentences looked equally plausible on the surface. Master's students were different. Once uninformed responses were stripped away, they clearly chose the correct sentences much more often. However, there were more cases of admitting they did not know than expected (PMID 27458424).

The authors of the original survey answered other critics again and stood by their claims. The alternative interpretations put forward by critics are correct in themselves. However, reading survey sentences that way is not an acceptable way of reading, and that interpretation says nothing about where the true value lies (PMID 26620955).

How much experts misunderstand this distinction is still under dispute like this.

Putting Parentheses to the Test in an Article

Let us say you encounter two news articles with the headline, "New Preventive Drug Cuts Heart Disease Risk in Half." Both state that a disease previously affecting 20 out of every 1,000 people over the course of a year has dropped to 10. The answers to both "Out of how many?" and "For how long?" are right there. At the end of each article, parentheses are included. Article A reads, "10 fewer per 1,000 (95% confidence interval 7 to 13 people)," while Article B reads, "10 fewer per 1,000 (95% confidence interval 1 more to 21 fewer people)."

The first question is, "What is the smallest possible estimate?" For A, even at its smallest, 7 fewer people per 1,000 get sick. For B, at its smallest estimate, 1 person more per 1,000 gets sick. Even the harmful side is included in those parentheses.

The second question is, "What is the largest possible estimate?" For A it is 13 people, and for B it is 21 people. Wide parentheses stretch further in both directions—toward the small end and the large end. Therefore, if you dismiss B as "no effect," you also erase the possibility that it had a huge effect.

Looking at these two endpoints also reveals whether the parentheses contain "no difference." Article A does not contain it, while Article B does. Translated into p-values, Article A is "statistically significant" and Article B is not. What if an article only reported that it was "statistically significant"? You would know that the parentheses missed "no difference." However, you would not know if the smallest estimate is 7 people, or much fewer than that.

If you wonder why B's parentheses are so wide, you look for how many people were recruited. In this example, Article A enrolled 10,000 people per group, and Article B enrolled 1,000. If an article lacks parentheses entirely, you cannot tell how much that number might bounce around.

One thing cannot be known from parentheses alone. Parentheses only capture the fluctuation that comes from who happened to be chosen by chance, and they do not tell you whether a study was biased. Bias is an error caused by the design of the study, not by chance. It happens when the two groups being compared were different to begin with, or when the way people were selected or measured was flawed (PMID 29531023).

In studies that estimate drug effects using observational data like hospital records, the errors typically written as numbers account only for random fluctuation. This fluctuation gets closer to zero as the dataset grows larger, but the error coming from bias does not shrink no matter how many people you gather (PMID 29531023). Therefore, the larger the dataset, the more important the errors outside the parentheses become.

The convention of 95% is also an agreement that holds true only when a study is free of bias. The same paper, which divides errors into these two types, reports that the original property of a 95% interval—capturing the true value 95% of the time—is restored only after adjusting the parentheses to account for bias (PMID 29531023).

Trials conducted carelessly tend to exaggerate effects and yield small p-values because of bias (PMID 38219106). What skews numbers is covered in "If People Who Take It Live Longer, How Were They Different to Begin With? — Confounding".

When you encounter parentheses, asking "What is the smallest possible estimate?" and "What is the largest possible estimate?" reveals how much that number might bounce around.

Where to Look Next

This article stopped at reading a single set of parentheses. The next part of the story is continued in "A Confidence Interval Is Not a Margin of Error — How to Read the Two Numbers in the Parentheses". One of those points is that even if two sets of parentheses overlap, you cannot conclude there is no difference (PMID 30119088). Which end of the parentheses to look at in a trial showing a new treatment is "not much worse" than the standard treatment, and whether the bars drawn in a figure are parentheses or not, are also in that article. That article covers in more detail the disputes sparked by experts' misunderstandings.

What the p-value is precisely the probability of is examined in "What Is p = 0.03 the Probability Of? — The One Question a p-Value Answers", and how statistical significance depends on the results of how many people is examined in "Where Did 0.05 Come From? — Counting How Many Patients a Finding Rests On". How to read the parentheses of multiple studies gathered in a single figure is covered in "A Meta-Analysis Does Not Take an Average — How the Diamond in a Forest Plot Is Built".

Other articles in "Numbers from Scratch" are "If the Risk Was Cut in Half, How Many Fewer People Got Sick?", which asks how many people a "50% reduction in risk" translates to; "What Does "Statistically Significant" Actually Say?", which asks what "statistically significant" means; and "If People Who Take It Live Longer, How Were They Different to Begin With?", which deals with confounding—the fact that moving together does not mean causation.

Source Papers

• "Making meaningful inferences about magnitudes." (2006) — PMID 19114737 — Sample studies provide only an approximation of the true value (the population value). It proposes using confidence limits to capture a plausible range for the true value, and judging the result as inconclusive if that range spans both practically beneficial and practically harmful values, or else judging it in the direction of the observed magnitude (proposal paper).
• "Knowing how effective an intervention, treatment, or manipulation is and increasing replication rates: accuracy in parameter estimation as a partial solution to the replication crisis." (2021) — PMID 32378428 — With sample sizes common in psychology, the sample distribution of the effect size is very wide, meaning that even in a perfect replication, the estimated value can vary considerably from sample to sample. It concludes that much larger studies are needed.
• "Trend in randomized controlled trials." (2003) — PMID 15526517 — A non-significant "trend" may disappear or reverse direction if the sample size is increased. To discuss a trend, one must interpret the confidence interval.
• "Editorial Commentary: The Statistical Fragility Index of Medical Trials Is Low By Design: Critical Evaluation of Confidence Intervals Is Required." (2024) — PMID 38219106 — A confidence interval is the range of values compatible with the data, its width indicates the precision of the result, and the values within it represent room for clinical importance. Sloppily conducted trials are prone to yielding large effects and small p-values due to bias (editorial commentary).
• "[Statistical results: which method of presentation to chose?]." (1997) — PMID 9172963 — Estimation and confidence intervals reveal the direction and magnitude of an association, and the narrower the interval, the more precise the estimate. This method also has limitations and the risk of misinterpretation does not disappear, leading some experts to suggest presenting both methods together in the absence of a consensus.
• "A primer on confidence intervals in psychopharmacology." (2015) — PMID 25742216 — A non-mathematical introduction to confidence intervals. It covers the interpretation of narrow and wide intervals, factors influencing width, and how to determine statistical significance using intervals, concluding that confidence intervals provide far more information than p-values when drawing conclusions.
• "Obtaining power or obtaining precision. Delineating methods of sample-size planning." (2003) — PMID 12971200 — It presents a method for determining sample size so that the expected width of the confidence interval is sufficiently narrow (from the perspective of accuracy in parameter estimation), alongside the perspective of statistical power.
• "Setting sample size to ensure narrow confidence intervals for precise estimation of population values." (2011) — PMID 21336224 — Sample sizes determined by desired statistical power and expected effect size are often too small to provide an interval narrow enough to precisely estimate the population value.
• "Precision and Sample Sizes Achieved for Infant and Young Child Feeding Indicators Evaluated in Anthropometry Assessments: A Secondary Analysis of Population-Representative Surveys in Refugee Settings." (2026) — PMID 40803324 — 203 surveys from refugee settlements across 15 countries between 2013 and 2019. Breastfeeding experience indicators included all children aged 0–23 months, yielding a large sample and a proportion close to 100%, making them the most precise (median half-width 3.1%), whereas the indicator for exclusive breastfeeding under 6 months was 12.6%, showing low precision. It concludes that great caution is needed when interpreting past surveys of such indicators.
• "Sample size and precision of estimates in studies of depression screening tool accuracy: A meta-research review of studies published in 2018-2021." (2022) — PMID 35362161 — Depression screening tool accuracy studies from 2018–2021: Among 103 studies with available or calculable intervals, 58 (56%) had sensitivity interval widths of 21% or more. Most samples were too small to yield precise estimates, and the results were similar to previous surveys.
• "Hypothesis testing and effect size estimation in clinical trials." (1997) — PMID 9012613 — Significance testing deals only with the existence of the hypothesis that there is no effect and does not estimate the magnitude of the effect. It recommends that effect size estimation accompanied by confidence intervals captures all the information of the test while prioritizing clinical importance.
• "The P Value: What It Is and What It Is Not." (2025) — PMID 41250654 — Properly interpreted, the p-value provides useful information, but it should not be used as the sole criterion for inference, and transparent reporting of effect sizes, confidence intervals, and context forms a more reliable foundation (review).
• "Robust misinterpretation of confidence intervals." (2014) — PMID 24420726 — When 120 psychology researchers and 442 students were asked about 6 confidence interval interpretation statements that the research team had all classified as false, both groups endorsed an average of more than 3 of them. Researchers barely outperformed students who had received no training in statistical inference.
• "The Interpretation of Scholars' Interpretations of Confidence Intervals: Criticism, Replication, and Extension of Hoekstra et al. (2014)." (2016) — PMID 27458424 — Confidence intervals admit two interpretations, so misunderstanding cannot be inferred solely from agreement with certain items, and a forced-response design may have artificially created the results. In a replication adding 2 correct items, first-year students endorsed correct and incorrect items at equal rates, while master's students endorsed correct items more after filtering out guesses (and admissions of not knowing were also more frequent than expected).
• "Continued misinterpretation of confidence intervals: response to Miller and Ulrich." (2016) — PMID 26620955 — Response by the original authors: Even if the critics' alternative interpretations are correct, they do not constitute an acceptable reading of the survey questions, lack support in the data, and are restatements of definitions that do not imply the location of a parameter.
• "Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data." (2018) — PMID 29531023 — Errors in observational medical data are divided into random error arising from sample fluctuation and systematic error arising from confounding, selection bias, and measurement error. Usually, only random error is quantified, which converges to zero as data grows larger, whereas systematic error remains regardless of sample size. It reports that calibrating intervals using negative and positive controls restores the nominal property where the 95% interval covers the true effect 95% of the time.
• "A Tale of Confusion From Overlapping Confidence Intervals." (2019) — PMID 30119088 — Even if confidence intervals for the means of two independent groups overlap, a true significant difference is not necessarily ruled out. It recommends estimating the interval for the difference itself and avoiding the assumption that overlap means no difference.