I² Is Not the Size of the Scatter — How to Read Heterogeneity Statistics

Meta-analysis is an analysis that gathers the results of multiple studies addressing the same question and combines them into one. Meta-analysis papers frequently feature a number called I². What this number measures precisely is the topic of this article, and its meaning is unpacked step by step below.

In meta-analyses covering the proportion of people with a certain disease, namely prevalence, this number is unusually large. Across 134 prevalence meta-analyses, the median of I² was 96.9%, with the middle half falling between 90.5% and 98.7% (PMID 35088937). Read with the commonly used scale (anything above 50% is "high"), almost all of them represent "high heterogeneity." Heterogeneity refers to the degree toซึ่ง results differ across studies, but its exact meaning is unpacked in the first section.

The authors of the survey do not view these 134 papers as analyses that should be discarded. In this survey, I² was larger when a greater number of studies was included. I² was also large when the prevalence combined from multiple studies (the pooled estimate) was extreme, falling below 10% or exceeding 90%. Therefore, the authors conclude that a high I² value is not always synonymous with high heterogeneity. They also write that in prevalence meta-analyses, I² may lack discriminatory power, and advise interpreting it carefully by avoiding arbitrary thresholds. Their final recommendation is to focus on describing the range of expected values when discussing heterogeneity.

The authors mention two methods for describing that range. One of them is the prediction interval, which we will cover later, and out of the 134 papers, 3 (2%) reported that prediction interval (PMID 35088937).

"A Meta-Analysis Does Not Take an Average — How the Diamond in a Forest Plot Is Built" stops at the random-effects model. The random-effects model is a method of combining results under the assumption that "the true value may differ slightly from study to study." This article deals with the numbers that actually measure that "possibility of difference."

We will learn six new terms, each of which is defined where it first appears. Forest plots, weights, fixed effects, and random effects are terms covered in that article, so they are only briefly recalled here when needed. All names of diseases and treatments appearing below are examples just to show how to read the numbers, and are not judgments on whether a treatment is good.

---

There Are Two Kinds of Causes for Spread

Let us say six studies that asked the same question gave different answers. The reasons fall broadly into two paths. One is chance in the process of drawing samples. Even if six people flip the same coin a hundred times each, the number of times heads comes up differs every time. The other is that the studies are measuring slightly different things to begin with. The participants differ, the dose and duration of the intervention (the treatment or procedure tested in the study) differ, and the time and tool for measuring the outcome differ.

Heterogeneity is the difference in results that arises because the studies are measuring different things to begin with. This phrase is used in two senses. The spread measured in numbers is statistical heterogeneity, and the actual differences among the studies that created that spread are clinical heterogeneity.

A guideline created by expert consensus defines clinical heterogeneity like this. It is the variation in treatment effect that comes from differences in clinically relevant characteristics—that is, between-study variation coming from participants, the type or timing of outcome measurement, and the characteristics of the intervention. This guideline issued recommendations in twelve categories on how to investigate clinical heterogeneity (PMID 24004523).

The place where clinical heterogeneity arises is visible even in a single outcome measurement scale. NIHSS is a scale that scores the condition of acute stroke patients. The degree to which multiple evaluators give the same score when scoring the same patient is called inter-rater reliability, and in the NIHSS, this reliability differed from item to item. The motor function items were 0.85 to 0.90, higher than the remaining items (0.58 to 0.85). This value means that the closer it is to 1, the more the evaluators gave the same scores to each other (21 papers, 818 patients, 9,696 assessments, PMID 39240475). This means that even with the same scale, the degree to which scores diverge among evaluators differs depending on which item it is.

The authors' conclusion is balanced. NIHSS scores showed overall moderate to excellent levels of inter-rater reliability, and the place with the greatest heterogeneity was non-motor function assessment. Non-English versions, remote evaluation, and retrospective analysis (an analysis looking back later at already accumulated records) had small effects in terms of the heterogeneity of NIHSS scores (PMID 39240475).

The relationship between the two types of heterogeneity is not as solid as one might think. Statistical heterogeneity is a correlate of clinical and practical heterogeneity—in other words, simply a value that moves along with it—and that correlation can sometimes be weak (PMID 19018930). This means that the spread measured in numbers does not directly reflect the actual differences among studies.

The First Number, Q — Can the Scatter Be Explained by Chance?

Q is a test statistic used to ask: "If the studies were all actually measuring the same true value, is this degree of scatter likely to happen?" This test is the oldest way to handle heterogeneity. The answer comes out as only one of two things: "difficult to view it as such" or "can be viewed as such."

Therefore, Q has an inherent limitation. The Q test only tells the meta-analyst whether heterogeneity exists, and says nothing about its degree. To bridge this limitation and express the degree of heterogeneity as a number, I² in the next section was proposed.

I² does not completely escape Q's weaknesses either. A Monte Carlo simulation is a method of testing a method by creating virtual data countless times on a computer. Studies comparing the confidence intervals of the Q test and I² using this method conclude that I² is useful as a supplement to the Q test. However, a qualifier is attached. I² also has the same problem of low statistical power—that is, the ability to catch heterogeneity that actually exists—when the number of studies is small (PMID 16784338).

Q has one more weakness. It lacks an intuitive interpretation for clinicians, and its statistical power is often low. This is why many meta-analysts have moved on to indicators like I² (PMID 31234230).

The Second Number, I² — A Proportion, Not a Magnitude

I² is the proportion of total variation in study estimates that comes from heterogeneity. This is the definition from the 2002 paper that proposed the metric (PMID 12111919). Study estimates are the effect values produced by each study, and total variation is the overall degree to which those values are scattered.

The background to how this metric came about is as follows. The degree of scatter can be measured by estimating between-study variance, but doing so ties the interpretation to a specific treatment effect metric (such as an odds ratio or mean difference, the unit representing the effect). Tests for the presence of heterogeneity depend on the number of studies. Therefore, the authors derived three statistics—H, R, and I²—that, by design, are independent of the number of studies and the treatment effect metric. The authors' conclusion is that H and I² are particularly useful, and that published meta-analyses are better off reporting one or both of them rather than a test for heterogeneity (PMID 12111919).

"Independent of the number of studies" is a design criterion aimed at making it so. In actual journal samples, as in the prevalence survey seen earlier, I² is sometimes observed to be larger when there are more papers.

What follows directly from the definition is the core of this article. I² is a proportion. It is a value that approximates what share of the total variation comes from true heterogeneity rather than random error. A 2026 overview puts this point as follows. Although this metric has become extremely widely used, it is frequently misunderstood as an absolute measure of the amount of heterogeneity, which it is not (PMID 41992898).

The difference between a proportion and a magnitude is revealed by a single example. Effect size is the numerical value representing the effect produced by an intervention. Let us assume that the average effect size of an intervention is 50 points. What we want to know is whether that effect moves between 40 and 60 across groups, or whether it moves between 10 and 90. Only then can we judge whether it is useful. Even if we hear that I² is 50%, we have no way of knowing whether it is 40–60 or 10–90. The authors' prescription is concise. If you want to communicate the predicted range of an effect, simply report that range (PMID 28058794).

The I² value written in a paper is itself an estimate. Because I² can be subject to great uncertainty, when interpreting the degree of heterogeneity it is essential to look not only at a single point estimate but also at the range where that value is likely to lie (interval estimation) (PMID 38502022).

As Studies Gain More Participants, I² Can Rise Even for the Same Amount of Scatter

A little-known consequence follows from the fact that I² is a proportion. The denominator of the proportion, total variation, contains the chance variation within each study. As the number of participants in the studies increases, that chance variation decreases. Then, even if the true differences between studies do not change by a jot, the share of the variation that those differences occupy becomes larger.

A 2025 paper tackled this property head-on. As a simple example, the authors show that the I² statistic depends heavily on the sample size of each study. This is because I² measures heterogeneity among observed effect sizes. Therefore, to reduce the influence of sample size, they propose an alternative metric that directly measures heterogeneity among study populations (PMID 40405379).

The authors' conclusion is to use this alternative metric as an auxiliary indicator of heterogeneity. Its use is to screen for situations where the effect sizes of studies are actually similar, leaving almost no biological difference. In such a situation, a fixed-effect model—which assumes that all studies measure a single true value—may be appropriate. Yet, even in that very situation, if the sample size is large enough, I² can rise to its maximum value of 1, and the authors add that I² would then suggest using a random-effects model (PMID 40405379).

This story differs from the observation about the number of studies seen earlier (PMID 35088937). The former was about how many studies went into a meta-analysis, whereas here it is about the sample size of each study—in other words, how many participants are in a single study.

Where Do the Thresholds of 25%, 50%, and 75% Come From?

Papers often attach labels to I² values, such as "high heterogeneity with an I² of 70%." Those labels come from the classification in the Cochrane Handbook. The Cochrane Handbook is a standard guideline compiling methods for systematically collecting and combining multiple studies.

This classification has an issue where the intervals overlap with each other. The handbook's intervals are 0–25% for "low," 25–75% for "moderate," and 50% or more for "high." This means an I² of 50% is both "moderate" and "high" at the same time. Moreover, the primary literature provides only limited explicit theoretical or empirical justification for why these thresholds were chosen (2026 lineage-tracing paper, PMID 42359156).

The diagnosis by the authors of this paper is that current practice uses I² as the endpoint of interpretation rather than a decision-making tool for exploring heterogeneity. This means interpretation ends at labeling the number, and does not lead to decisions that dig deeper into the scatter. The prescription is a context-specific framework reflecting the type of outcome and the expected variation (PMID 42359156).

Applying thresholds mechanically can distort judgments. Frameworks that grade the certainty of evidence include a criterion to lower the grade if study results contradict one another (inconsistency). In a 2026 paper applying this framework to four meta-analysis cases, two of the cases had high I² values. However, visual inspection of the forest plots showed that point estimates were consistent, confidence intervals overlapped with one another, and all of the estimates lay on the same side of the decision threshold. Therefore, the authors determined that lowering the grade on the grounds of inconsistency was not justified (PMID 41796757). This means we cannot judge that study results contradict one another simply because I² is high.

The Third Number, τ² — It Tells the Magnitude, but Is Bound to the Scale

τ² (tau-squared) is the variance that indicates how widely the true values, which differ from study to study, are spread. Variance is a value that measures how scattered values are from the average. τ² is obtained by calculating this value from the distribution of "true values that differ from study to study" assumed by the random-effects model. While I² is a ratio, τ² directly measures the magnitude, so it tells us something I² cannot.

The price paid is the unit. When the degree of dispersion is measured as the between-study variance, the interpretation is tied to the specific treatment effect metric (PMID 12111919). This is just like how the number changes when you measure the dispersion of height in cm² versus in inch². Therefore, whether a τ² value of 0.09 is large or small is known only to someone who understands the scale that the value sits upon.

To bypass this property, indicators have also been proposed that express dispersion as a ratio compared to the effect being measured instead of an absolute amount. However, this method can produce very large values when the effect is small, making interpretation difficult (PMID 34328266).

τ² is also an estimate. τ, which is the square root of τ² , is not estimated very precisely when the number of studies is small or moderate. Moreover, the values estimated by the model (parameter estimates) and the values re-estimated by pulling each study's effect toward the overall average (shrunken estimates) can vary greatly depending on what the true value of τ is (PMID 38100240).

Then What Should We Look At — The Prediction Interval

A prediction interval is the range showing roughly where the true effect of a similar new study would fall if one were conducted right now. It answers a different question than a confidence interval. The confidence interval covered in "A Confidence Interval Is Not a Margin of Error — How to Read the Two Numbers in the Parentheses" is the range answering "where is the average roughly located?" Prediction intervals are often plotted alongside the diamond in a forest plot.

The argument in the paper that proposed routinely including this interval is that it is easy to read. How much the strength of the effect varies from study to study is captured in metrics like τ² or I², but those metrics do not have simple clinical interpretations. Prediction intervals are less complex because they directly present the range where the true effects are expected to lie in similar studies (PMID 27406637).

The authors constructed prediction intervals for actual meta-analyses. The subjects were the first meta-analyses of Cochrane Intervention Reviews from 2009–2013, and a 95% prediction interval was calculated for each. Looking at the 479 of them that were significant under a random-effects model (p<0.05) and had an I² greater than 0, in 72.4% of them, the prediction interval suggested that the intervention might have no effect or even be in the opposite direction. In 20.3% of those 479, the effect could be completely opposite to the point estimate of the meta-analysis (PMID 27406637). This means that a significant average effect and what kind of effect will appear in a new study in a different setting are entirely different questions.

The practice of classifying heterogeneity as low, moderate, or high using statistics like Q-values, p-values, and I² is ubiquitous. A paper summarizing the roles of these metrics notes that this practice is nevertheless incorrect. These statistics do not tell us how much the effect size actually varies, and classifications based on them are at best uninformative and frequently misleading. The statistic that answers how much the effect size varies is the prediction interval (PMID 37940120).

There is one exception to this. The fact that I² does not tell us the range of variation in effect size is an exception when I² is 0% (PMID 38938910).

Yet this interval is rarely included. The probability of prediction intervals being reported was 1.7% (0.9–3.3) in sports medicine and 3.9% (2.4–6.6) in medicine. This is the result of randomly scanning 1,500 meta-analyses from top sports medicine and general medical journals between 2012 and 2022, looking at 866 of them (514 in sports medicine) that used a random-effects model (PMID 38501202).

In 60% of the 220 sports medicine papers where prediction intervals could be directly calculated, the reported confidence intervals and the calculated prediction intervals pointed to different conclusions. The prediction intervals were 3.4 times wider than the confidence intervals. In this sample, this means that in more than half the cases, the conclusion reached by looking only at the confidence interval changed when viewed through the prediction interval. The authors' conclusion comes with a hedge. The widespread misinterpretation of random-effects meta-analyses could mean that potentially harmful treatments or treatments with insufficient evidence base are actually being used (PMID 38501202).

Prediction intervals also have a weakness. There is one requirement that a 95% prediction interval must meet. Every time one is created, it just needs to capture the true effect of a similar new study 95 times out of 100 on average. Yet in meta-analyses with a small number of studies, that "how many times it captured it" is scattered in a skewed pattern. Therefore, looking only at the average can easily lead to misunderstanding and misreading (a 2026 paper that exhaustively investigated calculation methods published to date through simulation, PMID 41992906).

Can You Rest Easy If I² Is Low?

Let me ask it the other way around. If $I^2$ is written as 10%, can you trust that meta-analysis? When checked through simulation experiments, the mere fact that $I^2$ is low does not let us know whether the results can be trusted (PMID 24320992).

Looking first at the part that matches expectations, as heterogeneity grew larger, $I^2$ grew larger and the probability of drawing correct inferences from the meta-analysis decreased. The parts different from expectations are two. First, when heterogeneity was at a low level, $I^2$ failed to predict the accuracy of the meta-analysis results. Second, when heterogeneity was at a high level, even meta-analyses with low $I^2$ values had a low predictive value—in other words, a low probability that their results were correct.

The most common situation in reality is one where the level of heterogeneity is unknown. At that time, if $I^2$ is low, the predictive value of the value produced by combining multiple studies (the pooled estimate) can be scattered widely, so it must be interpreted cautiously. The authors' concluding sentence is the answer to this section. Quantifying statistical heterogeneity with $I^2$ is helpful only when the amount of clinical heterogeneity is unknown and $I^2$ is high. The authors also write that an objective method to quantify the levels of clinical and methodological heterogeneity is urgently needed (PMID 24320992).

A commentary covering statistical tests of heterogeneity and bias overall points in the same direction. These tests are very popular, but their limitations are frequently not recognized. It is inappropriately and confidently implied that these tests give reliable answers to questions that are not statistical in nature. False positive signals that report a signal does not exist as existing, and false negative signals that miss an existing signal, can both be common (PMID 19018930).

An Attempt to Find the Source of the Scatter — Subgroup Analysis

Once you know there is heterogeneity, the next step is to ask why it arose. A subgroup analysis is a method of splitting studies by their characteristics and recalculating a summary value for each bundle. These are bundles like studies that only included children or studies that used high doses.

Heterogeneity is not infrequently left unreported. In heart failure meta-analyses, heterogeneity was reported in only 422 out of 612 results (68.9%) (126 papers from 11 top journals between 2009 and 2019, PMID 33131285). Of the reported results, 137 (32.5%) had I²=0%, meaning no observable heterogeneity. The rest were low (I²<25%) in 40, moderate (25~50%) in 76, and high (>50%) in 169 (40%). The interval used in this survey is different from the handbook interval we saw earlier.

The technique most commonly used to investigate the source was what this survey classified as sensitivity analysis (68 cases). A sensitivity analysis is a method of seeing whether a result holds up even if some studies are removed or conditions are changed. Next came subgroup analyses (59 cases) and methods examining the relationship between study characteristics and effects using regression equations (40 cases). The authors' conclusion is that although heterogeneity is an essential component, it was not reported in nearly 30% of results, and the handling methods vary wildly, so standardization is needed (PMID 33131285).

Most differences found through subgroup analyses fall short of being large differences. When the same treatment works differently in different subgroups, it is called an interaction. Among 327 meta-analyses pooling individual participant data, there were 187 statistically significant interactions. Among them, only 7 (3.7%) crossed the mark (a standardized effect difference of 0.8) where the results of two strata could be considered visibly split. Among the 42 where the authors claimed "this subgroup should be treated differently," not a single one crossed that mark (PMID 30445577). This means that being statistically significant is different from the difference being large.

There is balance in the authors' conclusion. When individual data are available, significant interactions appear in about one-third of cases. And a fair number of those can provide opportunities for stratified treatment decisions, meaning tailoring treatment for each patient stratum (PMID 30445577).

The consensus on how to investigate clinical heterogeneity is still thin. In a survey collecting 101 relevant guidelines, the data themselves generally had a low risk of bias, but the agreement among them was extremely sparse (PMID 22846171). Even so, there were recurring recommendations. Planned investigations should be specified in the review protocol (an analysis plan written in advance), clinical experts should be included on the team, and covariates (study characteristics put into the analysis to explain effect differences) should be chosen with clear scientific rationale while ensuring a sufficient number of trials per covariate. The recommendation to interpret the results cautiously is also repeated.

What to Ask Before the Numbers

1. What percentage of what is $I^2$, not just how many percent. It is the share of total variation that is heterogeneity, not how much the effect varies (PMID 41992898).
2. Is a prediction interval included. If not, that meta-analysis has not answered "what kind of result will show up in a similar new study" (PMID 37940120).
3. Are the studies large. Even if there is almost no real difference between studies, if each study's sample is large enough, $I^2$ can climb all the way to 1 (PMID 40405379).
4. Did they look into the source of heterogeneity, and how did they read that result. A significant subgroup difference is not automatically a large difference (PMID 30445577).

If you leave behind just one thing, it is this. $I^2$ is not the size of the scatter, but the ratio of the scatter's share that is not by chance. If you want to know the size, you must look at the prediction interval.

The Spaces This Article Left Blank

Heterogeneity metrics also have their uses. A paper comparing four metrics through simulation recommends two of them, including I², for measurement purposes. However, it notes that these metrics should be used as descriptive statistics with an intuitive interpretation from a clinical perspective—in other words, numbers describing the state of the data—and that one should not judge the significance of heterogeneity based on their magnitude alone (PMID 31234230).

In the same study, the statistical power of I² was quite close to that of the Q test. Of the four metrics, the two recommended showed good agreement with each other, but all other pairs were generally poor (PMID 31234230). This means that depending on which metric is chosen, the judgment of the same data can conflict. Neither the four presented here nor the three covered in this article (Q, I², τ²) are all the metrics in the world. Several others have been proposed as well.

How heterogeneity variance is estimated in the random-effects model and what that parameter precisely represents were only touched upon by name in this article (PMID 34550584). Unresolved problems already appeared in the previous section. The fact that statistical heterogeneity is low does not mean that clinical heterogeneity is low. And the final sentence of that simulation paper is that an objective method to quantify the levels of clinical and methodological heterogeneity is urgently needed (PMID 24320992).

There are also spaces this article intentionally left untouched. They are the studies that were not included in the forest plot in the first place. If there are studies that were not published because their results were lackluster, the numbers covered in this article are also values calculated over those vacant spots.

Those vacant spots are handled by another side. The funnel plot, which is a graph trying to detect those vacant spots, the phenomenon where smaller studies show larger effects, and how unpublished studies contaminate heterogeneity metrics themselves are covered in "Same Trials, Different Conclusions — Publication Bias and What Went Into the Pool". A chapter covering the random-effects model also discusses publication bias at the end as an alternative explanation for why observed effect estimates form a distribution different from expectations (PMID 34550584).

Source Papers

What Heterogeneity Metrics Measure

• "Quantifying heterogeneity in a meta-analysis." (2002) — PMID 12111919 — Measuring by variance between studies ties interpretation to a specific treatment effect index, and the test for existence depends on the number of studies. From a mathematical standpoint, H, R, and I² are derived independently of the number of studies and effect indices. I² = the proportion of total variation stemming from heterogeneity. The conclusion is that H and I² are particularly useful, making it better to report either or both of them rather than the test for heterogeneity.
• "Reflections on the I-squared index…" (2026) — PMID 41992898 — I² approximately measures the proportion of total variability coming from true heterogeneity rather than random error. It is extremely widely used, but often misunderstood as an absolute scale measuring the amount of heterogeneity, which it is not.
• "Basics of meta-analysis: I2 is not an absolute measure…" (2017) — PMID 28058794 — Example with an average effect of 50 points. With I²=50%, you cannot tell whether the effect is 40–60 or 10–90. The prescription is simply to report the predicted range.
• "Assessing heterogeneity…: Q statistic or I2 index?" (2006) — PMID 16784338 — The Q test only tells you whether heterogeneity exists or not and does not report its degree. The objects of comparison are the Q test and the confidence interval around I². In Monte Carlo comparisons, I² is useful as a complement to the Q test, but suffers from the same problem of low statistical power when the number of studies is small.
• "Comparison of four heterogeneity measures…" (2020) — PMID 31234230 — In a comparison of four metrics, the power of I² was quite close to Q, and I² and R̂_I showed good agreement with each other, whereas all other pairs were generally poor. The recommendation is to use those two, but not to judge significance based on magnitude alone and instead use them as descriptive statistics.
• "An Alternative Measure for Quantifying the Heterogeneity…" (2025) — PMID 40405379 — Demonstrates with a simple example that I² heavily depends on study sample size, and proposes an alternative metric independent of sample size. The conclusion includes that "even if effect sizes are actually similar with almost no biological difference, I² can rise up to 1 if the sample size is large enough."
• "Comparisons of various estimates of the I2 statistic…" (2024) — PMID 38502022 — I² cannot be viewed as an absolute scale and can be subject to great uncertainty, making it essential to look at its interval estimation together when interpreting the degree of heterogeneity.
• "On ratio measures of heterogeneity…" (2022) — PMID 34328266 — I² is the share of random-effects variance out of total variance. The coefficient of variation is the ratio of the random-effects standard deviation divided by the effect of interest, giving relative heterogeneity against the effect. However, if the effect is small, it can yield very large values, making interpretation difficult.
• "How trace plots help interpret meta-analysis results." (2024) — PMID 38100240 — When the number of studies is small or moderate, τ is not estimated very precisely, and parameter and shrinkage estimates can vary greatly depending on the true value of τ.

Thresholds and Labels

• "Reinterpreting I2 Thresholds…" (2026) — PMID 42359156 — Because the Cochrane Handbook classifications (0–25% low, 25–75% moderate, ≥50% high) have overlapping definitions, an I²=50% satisfies two categories at the same time. Primary literature provides only limited theoretical and empirical justification for thresholds. Current practice uses I² as a terminal point of interpretation rather than a decision-making tool. The prescription is a context-specific framework.
• "Misusing I2 for inconsistency…" (2026) — PMID 41796757 — A certainty of evidence assessment framework was applied to four meta-analyses. In two cases, I² was high, but upon visual inspection, point estimates were consistent, confidence intervals overlapped, and they all fell on the same side of the threshold, meaning downgrading for inconsistency was not justified. The remaining two cases had precision grades lowered due to insufficient sample size.

Prediction Intervals

• "Plea for routinely presenting prediction intervals…" (2016) — PMID 27406637 — In 72.4% of 479 Cochrane reviews from 2009 to 2013 with significant random-effects (p<0.05) and I²>0, prediction intervals suggested the possibility of no effect or a reversal in direction, and in 20.3% the effect could be entirely opposite to the point estimate.
• "Avoiding common mistakes in meta-analysis…" (2024) — PMID 37940120 — Q-values, p-values, and I² do not tell you how much the effect size varies. The practice of classifying heterogeneity as low, moderate, or high using them is ubiquitous yet incorrect, and that classification is at best uninformative and frequently misleading. The statistic that answers this is the prediction interval.
• "How to understand and report heterogeneity…" (2023) — PMID 38938910 — I² does not tell you the range of variation in effect sizes (with the exception when I² is 0%). Prediction intervals enable reporting such as "about 10% of studies show trivial-to-moderate, 50% large, and 40% very large effects".
• "Meta-analysis prediction intervals are under reported…" (2024) — PMID 38501202 — Out of 1,500 top-journal papers from 2012 to 2022, there were 866 random-effects models (514 in sports medicine). Reporting probability was 1.7% (0.9–3.3) in sports medicine and 3.9% (2.4–6.6) in medicine. Among 220 calculable papers, 60% showed a disagreement in conclusions between confidence and prediction intervals, and prediction intervals were 3.4 times wider. The qualifier for the conclusion is "this may mean a harmful treatment is being used".
• "Assessing the properties of the prediction interval…" (2026) — PMID 41992906 — A 95% prediction interval captures the true effect of a similar new study in an average of 95% of cases each time it is constructed. Examining all published frequentist methods through simulation shows that when the number of studies is small, the distribution of their coverage proportion has a non-ideal, asymmetric shape, so looking only at the average easily leads to misunderstanding and misreading.

What Low Heterogeneity Means

• "The effects of clinical and statistical heterogeneity…" (2014) — PMID 24320992 — A simulation study. When heterogeneity was low, I² did not predict the accuracy of the meta-analysis results, and when it was high, even meta-analyses with low I² had low predictive values. The conclusion is that I² does not reveal the amount of clinical heterogeneity and is useful only when I² is high, and that an objective method to quantify clinical and methodological heterogeneity is urgently needed.
• "Interpretation of tests of heterogeneity and bias…" (2008) — PMID 19018930 — Statistical heterogeneity is merely a correlate of clinical and practical heterogeneity, and that correlation can sometimes be weak. Both false-positive and false-negative signals can be common. (The discussion of bias from the same paper is not cited in this article — it belongs to "Same Trials, Different Conclusions — Publication Bias and What Went Into the Pool".)

Where Does Heterogeneity Come From, and What Does It Do?

• "Meta-analysis of prevalence: I2 statistic…" (2022) — PMID 35088937 — Median I² of 134 prevalence meta-analyses was 96.9% (IQR 90.5–98.7%). I² was larger when the number of studies was greater and when the pooled estimate was extreme (<10% or >90%). Three papers (2%) reported prediction intervals. The conclusion is that a high I² is not always synonymous with high heterogeneity, and that one should avoid arbitrary thresholds and focus on describing the range of expected values—advocating both prediction intervals and planned sensitivity analyses as the methods. Prevalence literature reviews increased tenfold over a decade.
• "Consensus-based recommendations… clinical heterogeneity" (2013) — PMID 24004523 — Clinical heterogeneity = variation in treatment effects arising from differences in clinically relevant characteristics (inter-study variation in participants, types or timing of outcome measures, or intervention characteristics). Recommendations categorized into 12 areas through a consensus process.
• "Investigating clinical heterogeneity in systematic reviews…" (2012) — PMID 22846171 — Collected 101 guidance documents. The risk of bias in the documents was generally low, but there was extremely little consensus among them. Repeated recommendations were protocol specification, involvement of clinical experts, scientific rationale for covariates, a sufficient number of trials per covariate, and cautious interpretation.
• "Heterogeneity in measurement of NIHSS…" (2025) — PMID 39240475 — 21 papers, 818 individuals, 9,696 assessments. Inter-rater agreement for motor function items was 0.85–0.90, higher than the rest (0.58–0.85). The conclusion is that overall agreement is moderate to excellent, the greatest heterogeneity is in non-motor function assessment, and the effects of non-English versions, remote assessment, and retrospective analysis were small.
• "Assessment of Heterogeneity in Heart Failure-Related Meta-Analyses." (2020) — PMID 33131285 — 126 heart failure meta-analyses (612 outcomes, 2009–2019) from 11 top journals. Heterogeneity was reported for 422 outcomes (68.9%). Among them, I² = 0% in 137 (32.5%), low <25% in 40 (9.5%), moderate 25–50% in 76 (18%), and high >50% in 169 (40%) — these intervals differ from the handbook intervals. Techniques used were sensitivity analyses (68), subgroups (59), and regression (40). The conclusion is that it is unreported in nearly 30% and standardization is needed.
• "How often can meta-analyses of individual-level data individualize treatment?" (2019) — PMID 30445577 — 327 individual participant data meta-analyses. Of 279 reporting subgroup analyses, 102 (36.6%) had significant interactions. Of 187 interactions (124 individual-level, 63 group-level), 7 (3.7%) had large differences between strata (standardized effect difference ≥0.8). None of the 42 for which authors claimed treatment should differ were large. The conclusion is that significant interactions emerge in about one-third, and a moderate number of those can offer opportunities for stratification decisions.
• "Assessing Heterogeneity in Random-Effects Meta-analysis." (2022) — PMID 34550584 — Methods for estimating the heterogeneity variance parameter in random-effects models, what that parameter represents, and how to statistically explore explanations for heterogeneity. At the end of the chapter, publication bias is discussed as an alternative explanation when observed effect estimates form a distribution different from expectations.